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development  of  a JPA  algorithm,  Including  personnel  aptitude  and  experience,, 
type  and  complexity  of  task,  type  and  complexity  of  equipment,  and  degree  of 
procedurallzatlon  required.  A conceptual  model  Is  presented  for  use  by  the 
JPA  community  In  cost  trade-off  analyses.  In  JPA  selection  algorithms,  and 
In  the  grouping  of  theoretical  trends.  The  report  also  presents  a theoretical 
base  for  use  of  memory  In  JPA;  previews  a theory  for  mixing  JPA  techniques, 
principles,  and  methodologies;  and  outlines  goals  for  future  JPA  research 
and  technology  efforts. 
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FOREWORD 


This  research  and  development  was  conducted  in  response  to  Mavy  Decision 
Coordinating  Paper,  Performance  Aids  Test  and  Evaluation  (NDCP-Z0828*-PN) , 
under  the  sponsorship  of  the  Director,  Naval  Education  and  Training  (OP*‘99). 

The  overall  objectives  of  the  NDCP  are  to  define  the  state-of-the-art 
In  Job  Performance  Aids  (JPA)  technology,  to  develop  a conceptual  model 
for  an  Integrated  J]?A-based  personnel  system  Including  cost  benefits  and 
trade-off  analysis,  to  test  the  JPA  concept,  and  to  quantify  performance 
Increments  and  cost  benefits  obtainable  for  various  applications.  The 
Immediate  effort  described  In  this  report  relates  to  Task  1 of  the  NDCP: 

(1)  the  systematic  review  and  organization  of  existing  JPA  techniques,  re- 
lated research  data,  and  various  applicable  principles  and  concepts,  and 

(2)  the  establishment  of  meaningful  discourse  within  the  JPA  research, 
development,  and  operatlottal  communities.  The  final  two  sections  of  the 
report,  on  conceptual  organization  and  new  research  directions,  are  directed 
to  the  JPA  community  at  large.  As  such,  comments  are  not  only  welcomed 

but  solicited  on  the  concepts  and  recommendations  presented. 

Appreciation  is  expressed  to  Dr.  John  Foley  of  the  Air  Force  Human  Re- 
sources Laboratory,  Dr.  Edgar  Shriver  of  Kinton,  Inc.,  Mr.  Theodore  Post  of 
BioTechnology,  Inc.,  and  Mr.  Tom  Elliott  o.:  Applied  Science  Associates  for 
their  assistance  In  providing  key  documents  that  were  useful  In  describing 
and  organizing  the  existing  JPA  technology  base.  In  addition,  their  con- 
cepts and  those  of  others  involved  in  performance  aiding  research,  such  as 
Mr.  Reid  Joyce  of  Applied  Science  Associates  and  Mr.  John  Klesch  of  the 
Army,  were  extremely  valuable  in  arriving  at  those  presented  in  this  report. 


J.  J.  CLARKIN 
Commanding  Officer 
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Problem 


Data  collected  over  thto  past  20  years,  prliaarily  by  the  military  services 
but  also  In  the  civilian  sector,  suggest  that  Job  Performance  Aid  (JPA) 
technology  offers  major  potential  payoffs,  in  personnel  performance  incre- 
ments and  cost  avoidance  throughout  the  personnel,  training,  and  maintenance 
comunltles.  Unfortunately,  the  lack  of  a well-formulated  conceptual  defini- 
tion of  JPAs  has  Impeded  the  advancement  and  utilisation  of  JPA  technology 
In  military  personnel,  maintenance,  and  training  programs.  This  has  create 
difficulties  In  retaining  researchers  and  practitioners  in  the  field,  has 
discouraged  the  development  of  a solid  theoretical  basis  for  JPAs’,  and  has 
contributed  to  underutilization  of  the  technology.  Moreover,  an  integrated 
approach  to  JPAs  in  personnel  systems  has  never  been  taken. 

Purpose^ 

The  Navy  Personnel  Research  and  Development  Center  is  conducting  a major 
effort  concerning  the  test  and  evaluation  of  JPAs  and  their  integration  with 
Navy  personnel  and  training  systems.  That  effort,  outlined  in  Navy  Decision 
Coordinating  Paper,  Performance  Alda  Test  and  Evaluation  (NDCP-Z0828-PH) , 
has  as  its  first  task  the  detailed  definition  of  current  JPA  technology. 

The  objective  of  the  present  study,  conducted  as  part  of  the  larger  task, 
was  to  systematically  review  lUid  organize  existing  JPA  techniques,  related 
research  data,  and  various  applicable  principles  and  concepts. 

Approach 

A literature  review  and  conceptual  analysis  were  condv'uted  to  assess  and 
organise  the  stat3-of-the-art  of  JPA  research  and  tschnology. 

Results 

In  the  present  study,  101  JPA  systems  and  techniques  were  classified 
under  five  categories:  (1)  format/content,  (2)  display  media,  (3)  applied 
training,  (4)  peripheral  test/dlagnpstlc,  and  (5)  delivery  systems.  Major 
factors  that  were  identified  a&  critical  to  the  development  of  a JPA  algorithm 
Included  (p  personnel  aptitude  :xnd  experience,  (2)  type  cp^lexlty  of , 
task,  (3)  type  and  complexity  of  equipment,  and  C4)  the  degree  of  procedurall- 
zatlon  required.  A conceptual  model  was  developed  for  use  by  the  JPA  co^ 
munlty  In  making  cost  trade-off  analyses,  writing  JPA  selection  algorithms, 
and  grouping  theoretical  trends.  The  study  also  examined  a theoretical  base 
for  the  use  of  memory  in  JPAs;  previewed  a theory  for  mixing  JPA  techniques, 
principles,  and  mcchodologles;  and  set  goals  for  future  JPA  research  and 
technology  effort>3. 

Conclusions 

I.  Several  format/dontent  type  JPA  systems  are  sufficiently  developed 
to  be  Integrated  with  major  military  personnel  and  training  systems  with 
reasonable  expectations  of  major  payoffs  in  reduced  personnel,  training, 
and  maintenance  costs  and  in  increased  overall  operational  readiness. 
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2.  Wlehin  fornat/contcnt  types.  It  appears  that  three-^the  ^JFA,  the 
hybrid  aid,  and  the  deductive  aid— would  be  sufficient  to  cover  siost  opera- 
tlon/malntalner  aiding  needs  throughout  their  Navy  careers.  However,  the 
hybrid  aid  requires  further  reflnenent  before  full-scale  denonstration  or 
Implenentatlon. 

3.  Satisfactory  aiding  techniques  exist  for  all  nontrouhleshootlng 
operation  and  nalntenahce  tasks  and  for  the  aajorlty  of  troubleshooting 

tasks  on  laost  mechanical,  electrical,  and  electromechanical  systems,  but  ' 

no  siding  techniques  were  found  that  were  satisfactory  for  complex  dlgltel 
electronic  systems . 

i 

4.  The  greatest  weakness  In  JPA  systems  technology  Is  In  the  timely 
and  accurate  production,  distribution,  and  update  of  JPAs. 

5.  The  major  aid  development  coat  Iteoa  to  be  considered  when  com- 
paring one  format/content  type  JPA  with  another  are  (a)  the  detail  required 
for  front'^end  analysis  and  (b)  the  amount  of  graphic  or  verbal  enrichment 
necessary  for  Increased  Intelligibility. 

6.  The  greater  the  amount  of  procedurallratlon  and  entlchnect,  the 
greater  the  production,  distribution,  and  storage  costs  because  of  Increased 
aid  site. 

7.  Microform  And  audiovisual  systems  either  show  no  differences  or 
are  Inferior  to  hard-copy  print  media  In  terms  of  their  effects  on  user  per- 
formance. For  Information  aiding  needs,  most  of  the  unique  demands  of  the 
job  environment  (e.g.,  work  space,  wind,  temperature)  can  be  met  with  modified 
hard-copy  print. 

8.  Devices  that  can  increase  the  performance  capability  of  the  user 
beyond  that  possible  with  hard-copy  print  need  further  research  and  develop- 
ment. 

9.  The  FPJPA  appears  unsatisfactory  for  aiding  lower-aptitude  personnel 
to  perform  troubleshooting  tasks,  but  it  can  allow  Inexperienced  personnel 
who  meet  normal  aptitude  requirements  to  perform  such  tasks  with  minimal 
training. 

10.  Existing  JPA  development  and  selection  techniques  have  not  provided 
sufficient  organization  of  the  technology  to  permit  confident  research  or 
Implementation  decisions. 

11.  Basic  research  Is  needed  to  answer  questions  about  the  behavioral  , 

foundation  of  aiding'. 

12.  There  is  still  a need  to  determine  the  ef fleets  of  aiding  on  lowered 
aptitude  levels  and  the  motivation  to  use  aiding  systems. 

13.  Cost/performance  criteria  and  basic  cost  data  are  needed  to  make 
valid  cooq>arlson6  of  aids  In  trade-off  models. 
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14.  Research  Is  needed  to  help  deterslne  what  kind  of  process  is  re- 
qulrifed  to  produce  and  'aeasure  high-quality  JPAs. 

Reconmendatlons 


1.  The  OMsthods  and  problens  of  integrated  JPAs  with  training  and  job 
design  'ihould  receive  Ise^late  attention  In  efforts  directed  toward  algorltlm 
develcpnent. 

2.  Problens  of  carebr  advanceaent^  skills  retention,  and  other  najor 
personnel  systen  factors  should  be  investigated  to  detemlne  the-  effects  of 
progressively  changing  JPA  systems. 

3.  Task-oriented  training  and  job  design  should  be  developed  to  support 
or  conplement  the  aiding  systen. 

4.  Aiding  as  a personnel  systen  technology  should  act  on  an  equal 
footing  with  selection  and  training,  but  It  should  be  the  prJnary  technique 
for  aeetlng  aust  job  objectives  for  new  or  flrst-tem  enlistees. 

5.  Front-end  analyses  connonly  conducted  for  Integrated  JPA  and  train- 
ing requirenents  should  be  expanded  to  Include  job  design  requlrenents. 

6.  Selection  of  nedla  othet  than  hard-copy  print  should  be  based  on 
system  support  requirements  that  inprove  the  timeliness  or  accuracy  of  the 
aid  rather  than  personnel  performance-related  criteria. 

7.  Development  of  new  aiding  devices  (e.g.,  hand-held  calculators, 
holographic  displays),  aids  that  change  with  pernonnbl  needs  (e«g.,  hybrid 
aids),  and  aids  that  specifically  addreiss  digital  electronic  systeiss  ere 
needed  to  fill  gaps  and  to  advance  the  state-of-the-art  din  aiding  technology. 
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INTRODUCTION 


Problem 


The  Navy  currently  faces  expanding  personnel  costs;  decreasing  force 
levels;  declining  entry-level  skills;  ^nd  degraded  operational  readiness 
with  respect  to  personnel,  systems,  and  equipment.  Advances  In  personnel 
technology  are  therefore  being  combed  for  systems  or  techniques  to  enhance 
the  performance  of  lesser-trained  or  lesser-skilled  individuals,  to  reduce 
the  attrition  of  personnel  with  Navy  career  potential,  and  to  reduce  per- 
sonnel and  training  costs. 

Data  collected  over  the  past  20  years,  primarily  by  the  military,  suggest 
that  job  performance  aiding  technology  offers  major  potential  in  personnel 
performance  Increments  and  cost  avoidance.  Unfortunately,  the  lack  of  a 
well-formulated,  conceptual  definition  of  job  performance  aids  (JFAs)  has 
impeded  the  advancement  and  utilization  of  JPA  technology  in  military  per- 
sonnel, maintenance,  and  training  programs.  This  has  created  difficulties 
In  retaining  researchers  and  practitioners  in  the  field,  has  discouraged 
the  development  of  a solid  theoretical  basis  for  JPAs,  and  has  contributed 
to  underutilization  of  the  technology. 

Recently,  users  have  perceived  their  major  problem  to  be  that  of  select- 
ing from  among  the  hundreds  of  JPA  candidates,  and  researchers  have  obligingly 
directed  most  of  their  efforts  toward  the  development  of  JPA  selection  algo- 
rithms. Little  progress  has  been  made  In  this  area,  however,  because  a 
total  Integrated  approach  to  JPAs  in  personnel  systems  has  never  been  taken 
(Booher,  1977a).  Thus,  the  first  steps  for  the  development  of  a JPA-rele- 
vant  Integrated  personnel  system  are  O ) the  formulation  of  a conceptual 
definition  of  JPAs  and  (2)  an  analysis  of  the  state-of-the-art  in  performance- 
aiding  R&D. 

Purpose 

The  Navy  Personnel  Research  and  Development  Center  is  conducting  a major 
effort  concerning  the  test  and  evaluation  of  JPAs  and  their  Integration  with 
Navy  personnel  and  training  systems.  That  effort,  outlined  in  Navy  Decision 
Coordinating  Paper,  Performance  Aids  Test  and  Evaluation  (NDCP-Z0828-PN) , 
has  as  its  first  task  the  detailed  definition  of  current  JPA  technology. 
Specific  objectives  of  LliaL  task  are  (1)  co  conduct  a comprehensive  survey 
of  current  JPA  systems  and  techniques,  (2)  to  investigate  implications  for 
equipment,  environmental,  personnel,  and  behavioral  characteristics  of  job 
tasks,  and  (3)  to  develop  schema  that  would  aid  In  achieving  optimal  utiliza- 
tion of  state-of-the-art  JPA  technology. 

The  purpose  of  the  present  study  is  to  meet  these  objectives  by  systema- 
tically reviewing  and  organizing  existing  JPA  systems  and  techniques;  JPA 
research  data;  and  related  principles,  methods,  and  concepts. 

Background 


In  February  1977,  NAVPERSRANDCEN  sponsored  a conference  at  which  selected 
researchers  and  practitioners  were  asked  to  critically  assess  the  current 
status  of  JPA  technology  (Booher,  i977o),  A primary  objective  of  that 
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conference  was  to  determine  if  JPA  technology  could  be  structured  to  stimu- 
late meaningful  discourse  among  those  concerned  with  JPA  research,  develop- 
ment, and  implementation.  It  became  apparent  that,  while  such  a structure 
would  be  useful,  there  are  a great  many  unique  problems  associated  with 
JPA  technology  R^D.  The  conference's  findings  and  the  results  of  an  analysis 
by  NAVPERSRANDCEN  provide  the  basis  for  the  approach  tal'»n  in  this  study. 

JPA  Def  inition 


In  the  literature,  the  earliest  definition  of  the  JPA  was  expressed  in 
terms  of  man-machine  systems: 

A job  aid  is  an  item  whose  purpose  is  the  support  of  per- 
formances by  system  personnel,  which  are  necessary  for  over- 
all system  performance.  It  is  developed  as  an  alternative 
to  training  as  a means  of  obtaining  the  necessary  perfor- 
mances, and  is  therefore  designed  deliberately  to  complement 
training.  The  job  aid  is  developed  in  response  to  a require- 
ment which  specifies  the  performances  that  must  be  supported 
and  the  conditions  under  which  these  performances  must  be 
obtained.  It  is  an  end  product  of  a system  development  cycle 
which  is  delivered  as  part  of  an  operational  system.  (Wulff  & 

Berry,  1962,  p.  279) 

More  recently,  Smillle  (Mote  1)  redefined  the  Job  aid  more  concisely  as: 

Any  contrivance  which  permits  the  human  component  of  the 
system  to  perform  some  function  which  could  not  otherwise 
be  accoBq)lished  without  extensive  training  or  complex  infor- 
mation processing,  (p.  1) 

Most  often,  definitions  use  information  storage  and  retrieval  language. 

For  exanq>le,  Joyce  (1975)  said: 

The  defining  characteristic  of  a performance  aid  is  the 
capacity  to  store  information  for  later  retrieval  in  con- 
nection with  the  performance  of  a job.  The  aid  facilitates 
performance  by  reducing  the  memory — and  possibly  training- 
requirements  imposed  upon  the  performer,  (p.  5) 

Several  definitions  have  enq}hasized  pn?cedurallzatlon — typically  mean- 
ing fully  procedurallzed  job  performance  aids  (FPJPAs),  which  are  "Documents 
or  devices  that  give  precise  step-by-step  instruction  for  each  task  or  other- 
wise present  in  a concise  and  consolidated  manner  all  information  relevant 
to  the  task"  (Rowan,  1973,  p.  1).  This  is  primarily  a behavior-guiding  or 
i "active"  definition  in  that  It  states  the  direct  purpose  of  the  aid  in  terms 

I of  «mpportlng  the  efficient  performance  cf  a predetermined  set  of  behaviors. 

I 

I Some  researchers  prefer  to  look  also  at  the  indirect  or  "passive"  aspects 

of  job  aiding.  Chalupsky  and  Kopf  (1967),  for  example,  addressed  the  infor- 
' mational  content  and  instructional  aspects  of  job  aids  and  how  they  facilitate 
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performance  by  relieving  certain  Job  demands.  They  categorized  these  as 
task  modifying  aids  and  cited  as  an  example  a table  of  square  roots  that 
performs  the  appropriate  calculations  for  the  user. 

An  Interesting  theoretical  approach  to  defining  the  JPA  was  taken  by 
Lewis  and  Cook  (1969)  In  considering  the  JPA  ar.  undlrectlonal  communica- 
tion device.  In  describing  a "theory  of  telling,"  they  pointed  out  that, 
since  there  is  no  feedback  from  the  user,  the  design  of  a JPA  must  be  re- 
sponsive to  a model  of  the  user’s  abilities,  limitations,  and  processing 
functions  In  attending,  listening,  and  reading.  They  therefore  Inferred 
that  pictorial  and  linguistic  channels  are  primary  means  of  telling  and 
that  algorithms  in  the  form  of  quest lon-and-answer  flow  charts  or  question 
lists  are  superior  to  prose  for  this  purpose. 

A rather  unusual  insight  into  the  problem  of  defining  JPAs  is  the  recog- 
nition that  it  Is  not  the  media  or  format  aspects  of  JPAs  that  affect  per- 
formance aiding  so  much  as  it  is  the  ability  of  the  JPA  to  change  the  con- 
tent of  the  information.  Shriver  and  Hart  (1975)  concluded  that  much  of  the 
research  on  new  JPA  concepts  deals  with  the  software  link  between  the  hard- 
ware and  maintenance  personnel.  They  recognized  that,  traditionally,  this 
software  link  has  been  the  technical  manual.  Although  media  (e.g.,  micro- 
fiche, tape,  file  records)  have  been  substituted  in  the  link,  it  has  been 
Informational  content,  and  not  the  medium  of  presentation,  that  has  changed 
job  performance.  They  stressed  that  the  new  concepts  of  Importance  are  those 
concerned  with  changing  the  Information  that  flows  across  the  iJjik. 

Although  aspects  of  all  these  definitions  apply  to  JPAs,  the  following 
definition  has  been  adopted  for  the  purposes  of  the  present  research  on 
performance  enhancement:  "A  job  performance  aid  Is  any  device,  manual, 
guide,  or  tool  used  on  the  job  to  facilitate  performance  or  to  avoid  costs 
where  learning  from  the  aid  is  incidental."  Such  aids  can  vary  from  a simple 
metric  conversion  card  to  a complex  troubleshooting  logic  network  diagram 
used  in  electronic  fault  isolation.  Other  examples  are  a pilot’s  preflight 
checklist,  a hand-held  computer,  or  a job-guide  manual. 

The  written  (textual)  aid  could  be  a fully  proceduralized,  step-by-step 
guide  requiring  no  user  decision  making;  it  could  provide  only  an  overall 
picture  of  available  options  in  various  problem  situations  with  deductive 
guidelines  as  to  various  decision  paths;  or  it  could  be  a combination  of 
the  two,  directed  toward  an  intermediate  level  of  performance  aiding. 

For  the  most  part,  a JPA  is  a means  of  extending  human  capability  by 
providing  a mechanism  for  storing  information  for  later  retrieval  in  con- 
nection with  on-the-job  performance.  It  also  may  serve  as  a short-term 
memory  facilitator,  since  human  memory  often  is  not  sufficiently  reliable 
and  because  some  tasks  are  performed  too  seldom  for  habit  patterns  to  form. 
Consequently,  an  aid  may  reduce  or  eliminate  training  requirements  by 
eliminating  the  need  to  learn  and  retain  information  concerning  specific 
aspects  of  a job. 


DESCRIPTION  AND  COMPARISON  OF  JPA  TECHNIQUES 


The  g3.ossary  lists  101  JPA  techniques  that  were  surveyed  in  this  study. 
Shriver  and  Hart  (1975)  reviewed  many  of  them  in  the  development  of  their 
"fundamental  elements,"  noting  that  many  can  be  sorted  into  relatively  inde- 
pendent classes.  Post,  Price,  and  Diffley  (1976)  also  classified  various 
JFAs  into  basic  types  of  format  and  content  for  use  in  a guide  for  selecting 
formats  and  media  in  maintenance  information  presentation.  Considerable 
thought  has  been  given  to  the  problem,  but  no  general  agreement  has  been 
reached  among  JPA  investigators  on  a classification  system  that  would  be  most 
useful  in  describing  and  comparing  various  JPA  techniques.  This  study,  there- 
fore, has  sought  to  describe  JPA  technology  in  a global  sense,  considering 
aiding  teehniquss  not  only  individually  but  also  Sn  conjunction  with  other 
technologies  and  then  breaking  out  the  various  classes  of  aids  in  a way  that 
might  help  the  decision-making  activities  of  researchers,  developers,  and 
implementers. 

The  conference  on  job  aids  (Booher,  197 7n)  concluded  that  the  JPA  tech- 
nology base  can  be  organized  into  three  fairly  distinct  categories:  (1) 
technology  associated  directly  with  job-aiding  systems,  which  Includes  for- 
mat or  content  types,  display  media  types,  applied  training  systems, 
peripheral  diagnostic  systems,  and  delivery  systems,  (2)  technology  com- 
peting with  job  aids,  which  includes  automatic  test  equipment,  maintain- 
ability design,  formal  training,  selection,  and  human  engineering  design, 
and  (3)  technology  supporting  job  aids,  which  includes  instructional  system 
design,  job  design,  maintenance  simulation,  test  and  evaluation  technology, 
and  performance  testing.  This  chapter  simply  describes  and  compares  JPA 
systems,  devices,  and  techniques  that  exemplify  the  five  subcategories 
of  direct  job-aiding  systems.  The  glossary  lists  titles  and  acronyms 
commonly  used  to  Identify  most  of  the  JPAs. 

Format/Content  Types 


Table  1 classifies  37  JPA  techniques  according  to  (1)  basic  format  or 
content  types  (modified  from  Post,  Price,  & Diffley,  1976),  (2)  personnel 
qualification  requirements  for  using  each  technique,  (3)  the  type  of  hard- 
ware system  for  which  each  technique  is  designed,  and  (4)  general  demonstrated 
feasibility.  Middleton,  Gullllams,  and  Weber  (1977)  criticized  the  Special 
Case  formats,  pointing  out  that  they  can  generally  be  subsumed  under  the 
major  categories  for  troubleshooting  and  non troubleshooting  aids.  With  the 
exception  of  the  Quick  Fix  procedure,  the  analyses  performed  for  this  task 
confirmed  that  finding.  However,  the  Special  Case  categories  are  useful  in 
that  they  suggest  unique  requirements  (e.g.,  hazardous  conditions,  time,  one- 
trial  learning)  that  could  be  major  considerations  in  some  systems.  Also 
available  are  specifications  that  deal  Independently  with  certain  of  the 
Special  Case  aids  (i.e.,  maintenance  requirement  cards  and  maintenance  de- 
pendency charts)  and  that  give  them  a uniqueness  for  both  performance  and 
cost  considerations  that  could  be  lost  by  attempting  too  broad  a classifl- 
cf.tlon. 
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^Claimed  but  not  published  as  of  this  writing. 
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The  discussion  of  Post  et  al.  (1976)  on  the  Special  Case  aids  appears 
adequate  for  describing  the  unique  features  of  Troubleshooting,  Nontrouble- 
shooting, and  Special  Case  JPAs.  One  aid  type  that  they  did  not  discuss 
but  that  Is  listed  In  Table  1 Is  that  of  highly  complex  Integrated  systems, 
such  as  CMC,  CONSD,  FOMM,  SIMM  (listed  In  the  glossary  with  FOMM),  MDC, 
and  Work  Package.  The  CMC  and  CONSD  were  designed  only  for  experienced 
personnel,  while  the  FOMM,  MDC,  and  Work  Package  are  aimed  at  both  fully 
and  lesser-experienced  personnel.  Of  this  set,  only  the  SIMM  and  MDC  are 
supported  by  published  experimental  results,  and  those  results  have  not 
been  overly  favorable  toward  the  JPA,  In  some  cases,  they  have  even  been 
negative  (Rowan,  1973;  Shrlver  & Hart,  1975). 

The  FPJPA,  exemplified  by  Figure  1 (Westlnghouse,  1973),  Is  unique 
among  existing  JPA  format  or  content  types.  From  the  early  Army  studies 
until  the  present  time.  It  has  had  the  most  consistently  positive  experi- 
mental results.  Because  of  this,  the  FPJPA  has  been  mistakenly  regarded  by 
some  as  being  the  only  type  of  JPA.  It  forms  the  basis  for  the  JTM/JTG 
format  In  the  Army  ITDT  program  and  has  been  the  preferred  format  of  the 
Air  Force  In  Its  attempts  to  Implement  job  aids  on  the  C-141  program;  both 
services  have  prepared  specifications  for  procuring  both  nontroubleshooting 
and  troubleshooting  FPJPAs.  The  Navy  has  not  done  so,  but  It  does  have  the 
NAVAIR  Work  Package  specifications  and  the  NAVAIR/NAVSEA  writer  guidelines, 
which  Include  many  of  the  features  of  the  FPJPA  for  nontroubleshooting 
operation  and  maintenance  tasks.  The  Work  Package  concept,  which  is  clas- 
sified in  Table  1 as  a partially  procedurallzed  aid,  also  provides  for 
a troubleshooting  decision  logic  tree. 

Only  the  FPJPA  types  of  JPAs  (l.e..  Air  Force  FPJPA,  AMSAS,  C-141  Aids, 
JOBTRAIN,  OPTIMUM  PIC/WORD,  and  PIMO)  have  demonstrated  positive  results 
with  the  novice  technician  (acceptable  aptitude  but  Inexperienced).  Posi- 
tive results  have  also  been  claimed  for  the  Work  Package  concept  on  a 
demonstration  with  organization-  and  intermediate-level  maintenance  on  the 
APQ-126  radar  (Horn,  Note  2),  but  the  results  have  not  been  published. 

The  positive  results  of  the  MUC  (Krohn,  Clayton,  & Dunn,  1971)  with  In- 
experienced personnel  were  confounded  by  the  lack  of  an  experimental  con- 
trol, and  therefore  are  not  conclusive. 

The  FPJPA,  FOMM,  and  Work  Package  have  all  been  fully  developed  for 
electrical,  mechanical,  and  electromechanical  systems,  but  the  FOMM  generally 
is  not  considered  adequate  for  digital  electronics  or  mechanical  systems. 

As  shown  by  Table  1,  no  single  JPA  system  concept  includes  all  levels  of 
procedurallzation.  The  only  true  attempt  to  combine  two  or  more  levels  of 
proceduralization  was  the  AAT,  a NAVAIR  hybrid  aid  for  troubleshooting  that 
provided  two  separate  routes  to  successful  fault  Isolation.  This  technique 
Introduced  special  cues  designed  (1)  to  allow  the  technician  to  move  from 
one  route  to  the  other  and  (2)  to  suggest  why  certain  tests  were  being 
made.  The  latter  feature  was  to  encourage  greater  Incidental  learning  of 
higher  order  troubleshooting  skills. 
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Figure  1.  Example  of  FPJFA  foxrmat/content. 


The  discussion  of  Post  et  al.  (1976)  on  the  Special  Case  aids  appears 
adequate  for  describing  the  unique  features  of  Troubleshooting,  Nontrouble- 
shooting, and  Special  Case  JPAs.  One  aid  type  that  they  did  not  discuss 
but  that  Is  listed  In  Table  1 Is  that  of  highly  complex  Integrated  systems, 
such  as  CMC,  CONSD,  FOMM,  SIMM  (listed  In  the  glossary  with  FOMM),  MDC, 
and  Work  Package.  The  CMC  and  CONSD  were  designed  only  for  experienced 
personnel,  while  the  FOMM,  MDC,  and  Work  Package  are  aimed  at  both  fully 
and  lesser-experienced  personnel.  Of  this  set,  only  the  SIMM  and  MDC  are 
supported  by  published  experimental  results,  and  those  results  have  not 
been  overly  favorable  toward  the  JPA.  In  some  cases,  they  have  even  been 
negative  (Rowan,  1973;  Shriver  & Hart,  1975). 

The  FPJPA,  exemplified  by  Figure  1 (Wettlnghouse,  1973),  Is  unique 
among  existing  JPA  format  or  content  types.  From  the  early  Army  studies 
until  the  present  time,  it  has  had  the  most  consistently  positive  experi- 
mental results.  Because  of  this,  the  FPJPA  has  been  mistakenly  regarded  by 
some  as  being  the  only  type  of  JPA.  It  forms  the  basis  for  the  JTM/JTG 
format  In  the  Army  ITDT  program  and  has  been  the  preferred  format  of  the 
Air  Force  In  Its  attempts  to  Implement  job  aids  on  the  C-141  program;  both 
services  have  prepared  specifications  for  procuring  both  ncntroubleshootlng 
and  troubleshooting  FPJPAs.  The  Navy  has  not  done  so,  but  it  does  have  the 
NAVAIR  Work  Package  specifications  and  the  NAVAIR/NAVSEA  writer  guidelines, 
which  include  many  of  the  features  of  the  FPJPA  for  nontroubleshooting 
operation  and  maintenance  tasks.  The  VTork  Package  concept,  which  is  clas- 
sified in  Table  1 as  a partially  procedural Ized  aid,  also  provides  for 
a troubleshooting  decision  logic  tree. 

Only  the  FPJPA  types  of  JPAs  (i.e..  Air  Force  FPJPA,  AMSAS,  C-141  Aids, 
JOBTRAIN,  OPTIMUM  PIC/WORD,  and  PIMO)  have  demonstrated  positive  results 
with  the  novice  technician  (acceptable  aptitude  but  inexperienced) . Posi- 
tive results  have  also  been  claimed  for  the  Work  Package  concept  on  a 
demonstration  with  organization-  and  intermediate-level  maintenance  on  the 
APQ-126  radar  (Horn,  Note  2),  but  the  results  have  not  been  published. 

The  positive  results  of  the  MDC  (Krohn,  Clayton,  & Dunn,  1971)  with  In- 
experienced personnel  were  confounded  by  the  lack  of  adequate  experimental 
control,  and  therefore  are  not  conclusive. 

The  FPJPA,  FOMM,  and  Work  Package  have  all  been  fully  developed  for 
electrical,  mechanical,  and  electromechanical  systems,  but  the  FOMM  generally 
is  not  considered  adequate  for  digital  electronics  or  mechanical  systems. 

As  shown  by  Table  1,  no  single  JPA  system  concept  includes  all  levels  of 
proceduralizatlon.  The  only  true  attempt  to  combine  two  or  more  levels  of 
procedurallzation  was  the  AAT,  a NAVAIR  hybrid  aid  for  troubleshooting  that 
provided  two  separate  routes  to  successful  fault  isolation.  This  technique 
introduced  special  cues  designed  (1)  to  allow  the  technician  to  move  from 
one  route  to  the  other  and  (2)  to  suggest  why  certain  tests  were  being 
made.  The  latter  feature  was  to  encourage  greater  incidental  learning  of 
higher  order  troubleshooting  skills. 


Several  techniques  (e.g.,  ATOMS,  AVIS,  FOMM,  MAINTRAIN)  ate  sufficiently 
flexible  to  be  classified  as  both  simple  logic  deductive  and  system  descrip- 
tion types.  Some  (e.g.,  JOBTRAIN.  Work  Package)  even  provide  for  three 
levels  of  troubleshooting  flexibility,  but  none  were  found  that  mixed 
FPJPA  formats  with  the  other  basic  formats.  This  is  probably  due  to  two 
factors;  First,  FPJPA  is  a unique  form  of  aiding  that  relies  on  a complete 
front-end  analysis  and  fully  Illustrated  procedures  for  each  task  being 
aided  while  making  no  assumption  for  deductive  reasoning  abilities  or 
formal  training.  The  other  aid  forms  assume  at  least  a midlevel  technical 
background  achieved  through  training  or  experience.  The  second  factor  is 
that,  in  both  the  Army  and  the  Air  Force,  the  FPJPA  has  been  promoted 
as  a substitute  for  the  conventional  technical  order  or  technical  manual, 
not  as  an  adjunct  or  supplement  to  it.  It  was  thought  thaC  no  program 
manager  would  readily  approve  expenditures  for  two  parallel  aiding  systems 
on  the  same  equipment.  It  has  now  become  evident  that  no  existing  aid  can 
meet  all  the  objectives  of  cost,  performance,  and  manpower  of  a particular 
organlssation  at  the  command  level.  There  is  not  even  an  easy  division 
where  FPJPA,  FOMM,  or  the  Work  Package  can  be  used  that  will  be  consistent 
over  time  across  major  groups  of  programs.  There  are  instances  where  all 
13  aid  forms  may  be  necessary  cn  a single  system,  while  on  others  only 
one  may  be  required. 

Display  Media  Types 

Table  2 classifies  47  selected  JPA  techniques  according  to  the  7 basic 
media  display  types.  The  differences  in  types  of  display  and  special 
physical  features  are  much  easier  to  discern  than  are  the  format/content 
types,  but  it  is  probably  in  this  area  that  the  greatest  confusion  exists 
regarding  the  relative  advantages  and  disadvantages  of  perfonoance  aiding. 
There  is  little  evidence  to  support  the  contention  that  periormance  can  be 
further  Improved  beyond  that  obtainable  with  hard-copy  print  simply  by  the 
use  of  other  types  of  display  media.  However,  there  are  exceptions,  some 
quite  impressive,  that  will  be  pointed  out  in  the  following  discussion. 

Hard-copy  Print  Displays 

In  hard-copy  print,  the  major  physical  feature  of  interest  is 
size.  Printed  petlormance  aids  can  vary  from  hand-held  cards,  checklists, 
and  booklets  to  large  wall  charts  or  manuals  with  table  size  foldouts. 

For  presenting  operator  and  maintenance  instructions,  PIMO  has  short, 
step-by-step  Instructions  and  facing  illustrations  bound  in  a pocket-size 
booklet;  the  MDC  and  FLAP  formats  use  oversize  pages  so  that  complete  task 
Information  can  be  presented  on  a single  page. 

There  are  many  other  uses  for  hard-copy  print  JPAs  Chan  just  opera- 
tor and  maintenance  procedures,  particularly  in  the  smaller  formats.  For 
example,  Chalupsky  and  Kopf  (1967)  described  hard-copy  devices  used  as  aids 
in  product  inspection  tasks,  as  management  aids,  as  nomographs,  and  as 
conversion  charts.  Conversion  charts  in  particular  have  been  especially 
useful  in  patient  care  for  such  things  as  making  conversions  from  the 
apothecary  to  the  metric  system.  Hooprlch  and  Stelnemann  (1965)  found  that 
a Wallet-size  electrical  measurement  chart  raised  measurement  conversion 
performance  of  electronics  technicians  with  little  or  no  training  to  a level 
equal  to  that  of  experienced  technicians. 
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Table  2 


Display  Media  Typesi  Special  Physical  Features, 
and  Feasibility  Factors  Applicable  to  Various  JPA  Systcns  and  Techniques 
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There  have  been  other  physical  features  of  Interest  In  the  print 
format — for  example^  tab  size  (Rees  & Kama,  1959);  tabular  displays 
(Wright  & Fox,  1972);  and  special  page  coatings  to  resist  grease,  dirt, 
and  dampness — but  design  questions  apart  from  size  have  more  frequently 
concerned  the  desirability  of  embellishments.  Usually,  attempts  to  em- 
bellish a printed  aid  employ  color  or  cartoons.  If  such  changes  (e.g., 
color  coding  for  search  aiding,  organization  improvement,  or  reduction 
in  reading  requirement)  enhance  coiaprehenslon,  then  they  may  be  desirable, 
particularly  with  low-aptitude  groups;  but  when  used  strictly  for  their 
attention-getting  properties,  they  do  not  usually  increase  acceptance  or 
improve  performance.  AVIS  experimented  with  a female  voice  as  an  audio 
embellishment  but,  rather  than  improving  performance,  it  proved  to  be  a 
distraction  and  tended  to  reduce  confidence  in  the  message. 

Audiovisual  Displays 

Audiovisual  (A/V)  devices  have  been  expected  to  play  a much  larger 
role  in  performance  aiding,  but  military  studies  specifically  designed  to 
compare  such  devices,  such  as  Audlscan  on  the  Air  Force  C-lAl  aircraft 
(Goff,  Schleslnger,  & Parlog,  1969)  and  AVIS  on  the  NIKE  missile  system 
(U.  S.  Army  Test  and  Evaluation  Directorate,  1964),  have  shown  performance 
aided  by  A/Vs  to  be  poorer  than  performance  aided  by  visuals  alone. 

The  implementation  of  A/V  devices  for  performance  aiding  has  not 
met  with  much  success  in  the  civilian  sector  either.  Chalupsky  and  Kopf 
(1967),  in  their  survey  of  12  firms,  found  that  the  installation  of  A/V 
devices  increased  rapidly  about  1960  but  by  1963,  few  remained  in  full- 
time service  as  on-line  aids.  Travers  (1964)  explained  that  an  A/V  device 
is  most  likely  to  be  of  value  when  the  rate  of  information  input  is  very 
slow,  so  that  the  user  can  switch  from  one  mode  to  the  other  without  be- 
coming overloaded  with  information. 

One  particularly  novel  A/V  device  (ELECTROCULAR) , developed  by  the 
Office  of  Naval  Research  in  the  early  1960s,  was  a lightweight,  head-worn 
unit  that  contained  a 35mm  filmstrip  and  an  audio  tape  carrying  operation 
or  rnainteni'nce  instructions  (Brown,  1964).  The  unit's  optical  system 
allowed  the  user  to  see  both  the  instructions  and  the  work  environment. 

The  advantage  of  this  device  was  not  that  it  increased  comprehension, 
due  to  the  combined  sensory  modality  presentation,  but  rather  that  it 
allowed  greater  flexibility.  The  device  was  most  useful  in  presenting 
maintenance  instructions  where  there  was  insufficient  space  for  larger 
displays  or  where  the  operator  or  malntalner  needed  to  use  both  hands 
or  to  be  mobile. 

Audio  or  Visual  Displays 

Audio  systems  and  visual  displays  (in  addition  to  hard-copy  print) 
have  been  more  successful  as  separate  aiding  devices  than  have  the  com- 
bined A/V  units. 

Audio  Systems.  JPAs  with  programmed  audio  instructions  on  tape 
have  been  reported  by  Chalupsky  and  Kopf  (1967)  to  be  highly  successful 
for  cable  harness  or  cable  board  wiring  in  the  electronics  industry;  how- 
ever, only  one  device  of  those  surveyed  (SADIE)  was  a purely  audio  JPA  for 
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a military  application.  This  device  consisted  of  a 10-lb.  (4.53  kg.) 
specialty  designed,  two-track  magnetic  tape  player  and  a hand-held  controller. 
One  track  contained  a series  of  detailed  troubleshooting  Instructions  while 
the  other  track  contained  the  troubleshooting  sequence  logic  in  digital  form. 

A "yes"  button  advanced  the  tape  to  the  next  Instruction;  a "no"  button,  to 
another  ste  i in  the  sequence.  Although  impressive  claims  were  made  for  the 
device,  it  xs  not  known  to  be  in  use  and  no  experimental  results  have  been 
reported  in  the  literature. 

Perhaps  the  best  use  of  the  audio  channel  has  been  in  real-time 
voice  communication,  ranging  from  a supervisor  instructing  a worker  directly 
to  the  use  of  a voice  link  connecting  one  station  to  another.  Examples  of 
the  latter  are  the  McDonnell  Douglas  FEFI-TAFI  and  the  Quick  Fix  proce- 
dures, in  which  the  pilot  or  an  in-flight  maintenance  crew  member  alerts 
a ground  station  to  a maintenance  need  by  means  of  a coded  signal. 

Visual  Displays.  Photos,  picture  guides,  slides,  CRTs,  3-D  models, 
and  plastic  overlays  already  have  been  employed  extensively  us  JFAs.  For 
example,  the  implosion  technique,  which  used  slides,  allowed  the  user  to 
form  perceptual  blueprints  of  how  mechanical  parts  fit  together  or  how  elec- 
trical networks  connect,  but  it  has  only  been  applied  to  training,  llie 
SHOCK  ACTION  technique  provided  picture  guides  for  tank  crewmen  to  use  as 
on-the-job  performance  aids.  The  MMS  system  employs  a computerized  CRT  for 
displaying  and  manipulating  the  Maintenance  Dependency  Chart  (MDC) . TRACE 
uses  a plastic  overlay  on  circuit  boards  to  show  test  points,  current  flow, 
and  test  data  to  simplify  the  testing  of  transitor  radios.  Overlays  have 
also  been  used  in  production  facilities  for  the  assembly  of  printed  circuit 
cards.  In  one  case,  the  arrangement  of  component  bins  was  Integrated  with 
overlays  to  eliminate  assemblers’  decisions  in  selecting  or  locating  com- 
ponents to  be  attached  to  the  printed  card. 

* 

Microform  Systems 

Microform  is  often  considered  an  information  transfer  medium  that 
can  be  substituted  for  hard-copy  print  or  A/V  devices.  Actually,  however, 
the  advantages  of  microform  systems  alone  are  not  really  competitive  with 
such  devices  on  the  same  aiding  factors.  While  print  and  A/V  devices  are 
primarily  directed  toward  aiding  information  comprehension,  microform  systems 
are  designed  primarily  to  ease  information  access.  All  such  systems  used  as 
job  aids  are  storage  and  retrieval  devices  that  provide  a more  convenient 
means  of  accessing  large  information  sources  at  the  work  site.  Decisions 
to  use  microform  systems  for  on-the-job  aiding  are  based  not  on  direct  user 
requirements,  but  on  such  criteria  as  reduced  storage  requirements  or  re- 
duced distribution  and  update  costs. 

No  studies  were  found  that  directly  compared  microform  display 
systems  to  both  A/V  and  print  displays  for  relative  advantages  in  job-orlenced 
training  or  job  performance.  However,  a few  experimental  studies  compared 
microform  systems  against  the  print  mode  on  training  success  criteria.  The 
inost  recent  of  these  (Rizzo,  1977)  compared  the  efficiency  of  microfiche 
versus  traditional  paper  copy  for  training  Navy  personnel  on  the  Basic 
Electricity  and  Electronics  (BE&E)  curriculum.  That  experiment  measured 
(1)  the  time  it  took  individuals  in  three  aptitude  groups  to  complete  14 
individualized,  self-paced  modules  and  (2)  errors  on  comprehensive  examina- 
tion tests.  An  Important  finding  was  that  the  lower  aptitude  group  was 
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adversely  affected  (more  than  25%  greater  training  tine)  by  the  microfiche 
medium.  These  results  are  similar  to  those  of  an  earlier  Air  Force  study 
(Grausnick  & Kottenstette,  1971)  that  found  no  advantages  with  microfilm 
for  any  aptitude  group  and  definite  disadvantages  for  lower-aptitude  per- 
sonnel . 

Novel  Displays  and  Bevlces 

While  the  results  with  audiovisual  and  microform  systems  have  been 
disappointing,  some  nonpaper  aiding  devices  appear  to  offer  considerable 
advantages  over  static  paper  displays.  Examples  are  (1)  devices  such  as 
holography  and  three-dimensional  models,  and  (2)  Interactive  devices  such 
as  hand-held  computers  and  slide  rules.  Frey  (1976)  found  the  holographic 
medium  to  be  as  good  as  or  superior  to  line  drawings  and  photographs  on 
assembly,  discrepancy  identification,  and  information  extraction  tasks. 

The  production  of  good  quality  holograms  is  still  an  art,  however,  and  no 
attempt  has  been  made  to  produce  them  for  use  in  schools  or  in  the  field. 

Interactive  devices  can  almost  be  considered  as  a distinct  class 
in  aiding  technology.  These  devices,  which  can  Include  slide  rules,  hand- 
held computers,  and  Interactive  CRT  displays,  are  much  more  dependent  on 
the  state-of-the-art  in  Information  handling  than  are  other  devices,  which 
tend  to  present  one-way,  relatively  fixed  instructions.  Chalupsky  and 
Kopf  (1967)  classified  these  as  task  modifying  aids  because  they  allow  the 
user  to  achieve  a desired  result  without  performing  all  of  the  steps  that 
normally  would  be  necessary.  Such  devices  are  contrasted  with  task  supporting 
aids  such  as  procedurallzed  instructions,  which  guide  task  performance  but 
result  in  the  same  behaviors  that  would  occur  without  an  aid. 

Recent  work  of  Interest  in  this  area  is  that  of  Burkett  and  Kruse 
(Note  3)  on  card-programni  ^ble  and  chip-programmable  pocket  calculators. 

They  claim  to  have  discovered  an  Incidental  learning  process  they  call 
"Thinking  Process  Transmutation."  In  this  process,  the  user  is  guided 
incidentally  into  reorganizing  his  own  thinking  processes  to  fit  the  cal“* 
culator's  logical  processes.  Thus,  the  user  seems  to  develop  a new  capa- 
bility to  organize  thoughts,  to  define  necessary  steps,  and  to  develop  the 
most  efficient  procedures  for  problem  solution.  The  potential  of  such  appli- 
cations seems  enormous  in  equipment  fault  isolation  or  in  operator  analyses 
in  command-and-control  decision  making,  but  only  one  study  has  been  found 
that  actually  applies  the  concept  to  job  aiding  (Rigney  & Towne,  1977) . 

One  unique  method,  called  the  Concentrated  Odor  Sensing  Technique 
(COST),  uses  the  sense  of  smell.  Developed  by  the  Illinois  Institute  of 
Technology,  the  technique  employs  up  to  128  different  odors  concentrated 
and  encapsulated  in  tiny  seeds  designed  to  be  melted  by  heat,  cracked  open 
by  pressure,  or  dissolved  by  a liquid,  thereby  releasing  the  odor.  As  an 
aiding  device,  the  capsules  could  be  used  on  electrical  circuits,  motors, 
pumps,  hydraulic  lines,  or  bearings  to  Indicate  trouble  areas. 

Applied  Training 

Of  the  systems  surveyed,  several  aid  on-the-job  performance  through 
applied  training  techniques.  The  Importance  of  discussing  such  systems 
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here  Is  that  these  techniques,  to  some  degree,  have  been  designed  for  Imple- 
mentation In  conjunction  with  Job  aids.  Training  designed  In  this  manner 
tends  to  be  most  compatible  with  job  aiding  and  has  many  of  the  same  goals. 
Including  (1)  the  reduction  of  total  class  and  Instructor  time,  (2)  Imparting 
only  those  skills  necessary  for  performing  a Job,  (3)  an  emphasis  on  proficiency 
rather  than  knowledge  tests,  and  (4)  greater  "hands-on"  experience. 

The  older  Army  systems  (e.g.,  FORECAST,  JOBTRAIM,  MAINTRAIN)  were 
specifically  designed  around  a combination  of  training  and  Job  aids  that 
would  require  less  training  time  than  conventional  courses.  Some  of  the 
Navy-developed  maintenance  training  concepts  (e.g.,  TASKTEACH,  HITIPAC, 

ISO)  seem  to  consider  Interactive  CAZ  and  maintenance  simulation  to  be  the 
heart  of  Improvements  In  Job  perform^ce,  with  prescriptive  aids  and  tech- 
nical documentation  being  Incidental  to  the  training  process.  Such  tech- 
niques need  to  be  placed  on  an  equal  footing  with  aiding  to  fully  achieve 
the  advantages  of  aiding  technology. 

JOBTRAIN's  training  objectives  included  "theory,"  but  such  Instruction 
was  Introduced  wnly  as  the  student  needed  It  to  solve  practical  problems. 

In  FORECAST,  the  cue-response  or  behavioral  type  of  task  analysis  was  intro- 
duced as  the  basis  for  both  training  and  Job  aiding.  In  FORECAST  and  JOB- 
TRAIN  the  emphasis  on  theory  also  became  more  systems  or  functionally  oriented 
than  In  the  past.  The  shift  In  orientation  toward  logical  approaches  to 
troubleshooting  greatly  reduced  the  need  for  knowledge  of  the  fine  points 
of  electronics  theory.  Theoretically,  this  trade-off  (preliminary  analysis 
with  documented  logic  versus  knowledge  training)  could  be  pursued  to  its 
natural  limit;  all  information  In  the  book  for  use  directly  on  the  Job 
and  no  Information  presented  beforehand  for  instilling  knowledge  In  the 
head.  It  has  been  recognized  for  some  tine,  however,  that  certain  informa- 
tion necessary  for  Job  performance  is  better  assigned  to  the  head  than  to 
the  book  (Chalupsky  & Kopf,  1967;  Chenzoff,  1973). 

Although  convenient  for  JFA/tralnlng  task  assignments,  the  distinction 
of  head  versus  book  does  not  necessarily  parallel  that  of  training  versus 
aiding.  Contractors  on  the  Army  ITDT  program  are  finding  that  many  of  the 
knowledges  and  skills  needed  to  do  productive  work  with  available  Job  aids 
can  be  acquired  through  the  book  (in  an  aiding  format).  One  effort,  for 
example,  reduced  the  training  requirement  for  tank  maintenance  tasks  from 
200  CO  IS  after  finding  that  the  majority  of  training  tasks  could  be  learned 
by  simple  procedurallzed  training  aids  that  are  Indistinguishable  from  Job 
aids.  Many  of  the  remaining  skills  seem  to  be  learned  incidentally  while 
performing  with  Job  aids. 

There  appears  to  be  less  and  less  reason  for  the  separation  of  training 
and  aiding  In  the  Initial  stages  of  enhancing  Job  knowledges  and  skills. 

This  is  not  to  say  that  Job  aids  can  totally  replace  training.  Teel  and  Chancy 
(Note  4),  in  one  the  few  experiments  comparing  training  to  aiding,  showed 
that  combining  t..e  two  produced  almost  double  the  performance  lnq>rovement 
of  either  method  used  alone.  Furtheirmore,  as  Klesch  (1977)  pointed  out  with 
Jcb-orlented  training  programs  such  as  learner  centered  Instruction  (LCI) 
and  Job  aiding  programs  such  as  FIMO,  all  of  the  problems  of  personnel  careers 
(e.g.,  skills  development,  cross  training,  retention)  must  be  considered  If 
either  is  to  be  successful. 
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For  example,  neither  applied  training  not  fully  procedurallzed  job  aids 
adequately  address  the  long-term  personnel  problem  of  retaining  and  building 
a career  force  in  any  of  the  services.  This  then  leads  to  the  need  to 
consider  training  and  aiding  ns  changing  in  relative  usefulness  over  an 
individual’s  career. 

Peripheral  Test/Diagnostic  Systems 


Some  JPA  systems  combine  automatic  testing  diagnostic  features  vlth 
further  diagnostic  and  repair  instructions  for  maintenance  personnel  to 
use  in  correcting  malfunctions.  Such  systems  usually  cover  electronic 
equipment  and,  to  a lesser  extent,  hydraulic,  pneumatic,  end  other  flow 
systems.  Three  basic  types  of  aiding  systems  can  be  classified  under  this 
category:  (1)  built-in,  computer  aided  tests,  (2)  status  displays,  and 
(3)  fault-probability  displays. 


The  first  type  is  more  commonly  known  as  ATE  (automatic  test  equipment), 
but  to  be  classified  as  a JPA,  a system  must  be  capable  of  (1)  diagnosing 
the  problem  down  to  the  level  of  repair  and  (2)  providing  the  means  for 
aiding  the  repair  itself.  Examples  in  the  glossary  are  the  Westlnghouse 
ADMIRE/MIRAID  and  the  Vought  Aeronautics  CATA,  which  have  built-in  equip-  ' 

ment  sensors  that  provide  inputs  directly  to  a computer  for  first-level 

diagnosis.  The  test  equipment  then  provides  follow-up  instructions  for  I 

the  maintenance  technician.  Other  systems,  such  as  VAST  and  VATE,  are 
even  more  automatic  in  that  they  use  computerized  fault  Isolation  to  indi- 
cate which  unit  needed  to  be  replaced.  In  such  systems  an  aid  is  most  use- 
ful for  operation  and  maintenance  of  the  ATE  or  as  a backup  when  the  ATE 
malfunctioning. 


The  second  type,  "status  displays,"  either  key  lists  of  displays  to 
troubleshooting  instructions  or  include  a decision  logic  table  for  com- 
pleting the  aiding  function  (Post  et  al.,  1976).  A decision  logic  table 
may  be  preferable  where  a large  number  of  displays  are  present  since  it 
can  provide  patterns  of  display  readings  that  more  readily  guide  the  tech- 
nician to  the  trouble  areas.  A good  example  of  this  type  of  aiding  system 
is  the  FEFI-TAFI  system  on  DC-10  aircraft.  The  system  controls  and  dis- 
plays that  normally  are  provided  for  commanding  and  monitoring  the  aircraft 
or  the  operation  were  found  extremely  useful  in  indicating  system  failures. 
Earlier  studies  by  McDonnell  Douglas  had  found  that,  typically  (since  several 
indicators  and  conditions  sensed  by  the  flight  crew  are  associated  with 
specific  subsystems) , malfunctions  in  different  components  produce  relatively 
unique  patterns  of  normal  and  abnormal  indications  that  can  be  considered 
signatures  of  particular  subsystem  failures.  Figure  2 (Burrows,  Willets, 

& Miles,  1970)  illustrates  a typical  flight  engineer's  fault  isolation  (FEFI) 
decision  logic  table  of  indicator  patterns. 

The  final  type  of  aiding  system  in  this  general  category  is  that  of 
probabillty-of-fault  systems.  Usually,  these  attempt  to  use  historical 
data  of  system  usage  to  improve  reliability  prediction.  The  aid  allows 
a technician  to  go  directly  to  a fault,  based  on  major  trouble  symptoms 
and  reliability  prediction  data. 
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Peripheral  test/dlagnostlc  systems  are  most  useful  where  repairs  need 
to  be  done  rapidly,  as  on  an  aircraft  carrier  flight  deck  where  a quick 
turnaround  between  landing  and  launching  Is  needed.  Of  the  systems  sampled, 
those  that  have  attempted  to  facilitate  such  rapid  repairs  are  the  Grumman 
Aircraft  RAPIDS  and  the  REPOM/MIRM  system  sponsored  by  the  Naval  Ordnance 
Laboratory.  Although  very  helpful  In  rapid  fault  Isolation,  such  systems 
have  not  been  widely  implemented  because  of  the  high  cost  of  obtaining, 
maintaining,  and  analyzing  the  data  necessary  to  continuously  update  the 
maintenance  Instructions. 

Delivery  Systems 

Several  types  of  systems  that  are  classified  as  aiding  systems  are 
really  process  systems  for  developing  and  supporting  the  aid  Itself. 

Usually,  such  systems  provide  and  update  technical  information  to  opera- 
tors and  malntainers  and,  as  such,  are  outgrowths  of  the  various  technical 
manuals  (also  referred  to  as  technical  data,  documentation,  publications, 
and  orders)  systems  for  commercial  and  military  applications.  In  this  class 
of  systems  there  are  several  stages  In  the  process  that  could  contribute 
greatly  to  the  quality  of  the  final  aiding  product.  Many  of  the  JPA  systems 
identified  in  the  glossary  are  subsystems  of  complete  job  aid  delivery  systems. 
TRUMP,  for  example,  is  concerned  primarily  with  the  rewrite  and  update  of 
technical  publications,  while  MIARS  (listed  in  the  glossary  with  TRUMP) 
is  a microform  system  directed  toward  information  storage,  distribution, 
and  retrieval. 

The  two  systems  of  most  interest  in  this  discussion  are  the  Navy  NTIP 
system  and  the  Army  ITDT  program.  NTIP  is  concerned  with  each  phase  of  the 
delivery  process  but  places  considerable  emphasis  on  hardware  systems  for 
technical  information  production,  handling,  and  display.  The  ITDT  is 
primarily  concerned  with  supporting  both  the  publication  and  training  needs 
of  weapon  systems  and  equipment,  with  heavy  emphasis  on  a front-end  analysis. 

The  NTIP  has  identified  eight  necessary  stages  in  JPA  delivery  (Hughes, 
1977a,  1977b) — user-data  match  requirements,  data  acquisition,  content  genera- 
tion, content  capture,  content  replication,  distribution,  update,  and  feed- 
back— and  has  identified  some  research  needs  in  each  stage  (Figure  3). 

User-data  Match  Requirements 

The  user-data  match  stage,  in  specifying  the  appropriate  aid  formats, 
content,  and  media  for  any  particular  project,  considers  (1)  personnel  charac- 
teristics, (2)  the  results  of  the  analysis  of  the  equipment  or  system  and 
task,  and  (3)  environmental  constraints.  The  primary  media  considered  under 
this  system  are  paper  manuals,  A/V  devices,  microform,  and  video  disc  devices. 
This  stage  is  most  similar  to  (but  not  as  rigorous  as)  the  Army’s  front- 
end  analysis.  It  is  probably  at  this  stage  that  the  various  JPA  selection 
algorithms  should  be  used.  Determining  the  level  of  detail  required  and  the 
quality  of  the  task  analysis  (front-end  analysis)  to  be  performed  are  the 
most  critical  decisions  to  be  made  at  this  stage. 
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figure  3.  JPA  delivery  system  stages. 


Data  Acquisition 


Data  acquisition  Includes  the  specifications,  management  policies 
and  procedures,  and  data  acquisition  organizations  Involved  In  acquiring 
the  JPA.  In  addition  to  specifying  front-end  analyses  and  media,  attempts 
should  be  made  to  Introduce  quality  assurance  provisions  to  ensure  that  the 
delivered  JPA  Is  complete,  accurate,  comprehensible,  and  usable.  The  problem 
of  accuracy,  particularly  with  troubleshooting  JPAs,  appears  to  be  the  most 
difficult  to  solve  and  the  least  likely  to  be  Improved  by  Increased  rigidity 
In  the  specifications.  Braid  (1977)  suggested  that  this  problem  can  only  be 
solved  through  more  Intelligent  use  of  front-end  analysis  and  feedback. 

Content  Generation 


The  content  generation  stage  of  the  JPA  process  Includes  actually 
developing  the  content  to  be  displayed  with  a JPA.  Eight  steps  are  Involved; 
(1)  developing  or  obtaining  and  accessing  the  engineering  or  manufacturing 
data  base,  (2)  developing  planning  documents,  (3)  making  Interface  assign- 
ments with  training,  (4)  performing  maintenance  engineering,  provisioning, 
and  design  engineering,  (5)  conducting  the  front-end  analysis,  (6)  writing 
and  Illustrating  the  JPA,  (7)  validating  the  JPA,  and  (8)  compiling  program 
historical  data.  Regardless  of  the  final  user  medium,  the  end  product  of 
the  content  generation  stage  will  almost  always  be  a draft  JPA  on  paper. 

Content  Capture 

The  content  capture  function  transforms  the  draft  JPA  into  a form 
and  format  suitable  for  replication.  It  combines  the  functions  of  text  pro- 
cessing (usually  by  typewriter),  graphics  processing  (manually  prepared  or 
computer-assisted),  and  production,  with  the  output  being  a reproducible 
copy  In  a predetermined  medium.  This  entire  function  can  be  automated  with 
state-of-the-art  equipment.  TRUMP  is  one  of  the  most  sophisticated  military 
systems  for  this  function.  It  uses  optical  character  recognition  (OCR) 
to  scan  printed  pages,  with  graphic  Inputs  processed  In  photographic  form. 

Tlie  system  can  also  receive  alphanumerics,  edit,  and  make  data  correction 
Inputs  through  Interactive  terminals.  Outputs  can  be  either  microform  or 
reproducible  copy. 

Future  systems  are  expected  to  store  all  information  digitally. 

The  methods  for  digital  storage  that  have  been  considered  Include  magnetic 
tape,  magnetic  disc,  magnetic  drum,  optical,  semiconductor,  hologram,  and 
video  disc.  The  video  disc  and  hologram  appear  to  offer  the  most  promise 
for  mass  storage  techniques,  but  neither  Is  currently  suitable  for  applica- 
tion In  JPA  development  systems. 

Content  Replication 

Content  replication  Involves  printing  and  copying  technology. 

While  the  technology  for  copying  has  made  dramatic  progress  In  the  past 
years,  printing  technology  as  can  be  applied  to  JPAs  has  not  changed  appre- 
ciably for  many  years.  The  printing  process  Includes  several  steps  (i.e,, 
imaging,  stripping,  making  the  printing  plate,  printing,  and  binding), 
but  It  Is  only  In  the  binding  process  that  automation  Is  more  fully  developed 
for  JPAs.  It  Is  unlikely  that  JPA  printing  technology  for  delivery  systems 
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will  change  much  in  the  next  few  years  because  of  the  high  capital  invest- 
ment in  machines  now  being  used  and  because  JPAs  are  usually  short-run  but 
relatively  large,  especially  when  used  as  maintenance  information  aids. 

Improvements  in  copying  offer  more  possibilities  to  JPAs,  par- 
ticularly in  the  area  of  on-site,  on-demand,  quick-printout  capability 
and  in  the  reproduction  of  color  originals.  In  addition  to  present-day 
microform  copies,  it  is  expected  that  technical  information,  including 
graphics,  soon  could  be  delivered  in  digital  form  at  the  work  site,  where 
it  could  then  be  replicated  on  CRT  displays  or  produced  as  printed  copy. 

Distribution 


The  process  of  distributing  JPAs  for  complex  military  weapon  systems 
Implies  much  more  than  a stockpile  of  aids  that  can  be  delivered  to  user 
activities.  The  distribution  system  must  provide  for  (1)  initial  distri- 
bution to  JPA  users,  (2)  automatic  distribution  of  future  Issues,  changes, 
and  corrections,  and  (3)  special  one-time  requests.  In  military  systems, 
configuration  changes  are  common  due  to  continued  efforts  to  improve  equip- 
ment operability  and  reliability.  Also,  equipment  maintenance  often  turns 
out  to  be  somewhat  different  than  originally  planned  (e.g.,  planned  dependence 
on  automatic  test  equipment  that  becomes  unreliable).  JPAs  require  accuracy 
and  must  reflect  the  actual  equipment  being  operated  or  maintained  if  they 
are  to  function  properly.  Assigning  a configuration  or  identification 
number  to  the  JPA  for  administrative  control,  and  tracking  the  JPA  with  data 
management  techniques  throughout  its  life  cycle,  are  the  major  means  of 
ensuring  such  accuracy  and  fidelity.  At  present,  no  system  exists  outside 
of  the  technical  manual  or  technical  docun^entatlon  fields  with  the  means 
or  responsibility  for  performing  this  critical  function.  The  distribution 
process  also  must  maintain  (1)  adequate  supplies  for  rapid  response  to  re- 
quests and  (2)  an  archive  for  historical  reference  data  and  supply  stock 
replenishment. 

Feedback 


The  feedback  link  from  the  JPA  users  to  the  distribution  and  change 
systems  is  critical  to  JPA  effectiveness.  In  fact,  full  utilization  of  this 
link  may  be  the  only  way  that  troubleshooting  JPAs  (of  the  procsduralized 
variety)  can  be  made  to  satisfactorily  reflect  the  troubleshooting  routines 
needed  on  any  particular  system  or  equipment.  This  is  a particularly  weak 
link  in  current  technical  manual  systems — it  is  slow,  user  participation 
is  low,  and  too  little  direct  communication  occurs  between  the  user  and  the 
data  manager.  The  situation  is  not  likely  to  improve  until  the  user  is  con- 
vinced that  his  efforts  in  detecting  and  reporting  inaccuracies  in  the  JPA 
are  considered  seriously  and  expeditiously. 

Update 

The  update  subsystem  connects  the  configuration  control  and  feedback 
system  to  the  distribution  process.  It  has  content  capture  and  content 
generation  functions  (as  already  discussed)  for  original  materials,  but 
also  must  consider  methods  for  changing  original  material  and  for  inserting 
new  material.  Inherent  in  the  update  process  is  a methodology  for  deter- 
mining the  effects  of  a change  in  one  JPA  upon  other  data  and  logistic  ele- 
ments. When  a system  is  in  production,  there  should  be  provisions  for 
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coordinating  all  logistic  elements.  The  coordination  among  elements  for 
out-of-productlon  equipment  tends  to  be  under  the  control  of  Navy  facili- 
ties and  is  less  formalized,  but  the  problems  of  configuration  control 
are  considerably  reduced  by  the  time  an  equipment  reaches  this  stage. 

Here  again,  technical  data  systems  have  made  bona  fide  attempts  to  control 
and  manage  this  process,  but  it  still  is  done  less  efficiently  than  would 
be  required  for  a workable  JPA  system. 
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JPA  ALGORITHM  DEVELOPMENT  FACTORS 


In  the  development  of  a JPA  algorithm,  four  general  areas  of  JPA  tech- 
nology must  be  considered:  (1)  factors  Important  In  selecting  the  basic 
format  or  content  type  of  JPA,  (2)  requirements  for  special  (l.e.,  nonprint) 
media  or  special  physical  features  of  the  JPA,  (3)  the  relationships  between 
the  JPA,  training,  and  job  design,  and  (4)  JPA  test  and  evaluation  tech- 
nology . 

JPA  Selection  Factors 


In  designing  JPA  systems  to  meet  the  personnel,  training,  and  maintenance 
support  requirements  of  a hardware  system,  the  following  variables  are 
primary  considerations  In  selecting  the  basic  JPA  system  type: 

1.  Personnel  aptitude. 

2.  Personnel  job  experience. 

3.  Type  of  task  being  aided. 

4.  Complexity  of  the  task. 

5.  Complexity  of  the  equipment. 

6.  Type  of  equipment. 

7.  Degree  of  proceduralizatlon. 

Various  combinations  of  these  variables  have  been  Investigated  for  the  rela- 
tive effects  of  one  system  over  another  on  performance,  training,  or  job 
design. 

Personnel  Aptitude 

Rowan  (1973)  stated  that  the  JPA  and  training  literature  Is  replete 
with  studies  indicating  that  lower-aptitude  personnel  can  carry  out  non- 
troubleshooting maintenance  tasks  with  minimal  training  using  properly  pre- 
pared job  performance  aids.  Because  of  (1)  decreasing  aptitude  levels  among 
new  enlistees  and  (2)  a shrinking  manpower  pool  at  the  hlgh-aptitude  levels, 
JPAs  are  being  looked  to  as  a way  of  better  utilizing  lower-aptitude  per- 
sonnel . 


Overall,  however,  the  results  are  not  as  convincing  as  Rowan  has 
suggested.  While  numerous  studies  report  the  advantages  of  aiding  inex- 
perienced personnel,  few  directly  address  the  problems  of  lower-aptitude 
personnel.  Available  data  do  tend  to  support  Rowan’s  interpretation,  but 
even  the  best  studies  base  their  conclusions  on  a finding  of  no  statistical 
differences.  Moreover,  in  studies  that  found  no  differences  among  aptitude 
levels,  this  finding  was  usually  limited  to  nontroubleshooting  tasks  (Elliott, 
1967;  Hooprich  & Stelnemann,  1965).  Elliott,  for  example,  in  eoiiq)arlng 
proceduralized  job  guide  performance  for  20  high  school  seniors — 10  high 
aptitude  (80-95  on  the  AQE  Electronic  Index)  and  10  medium  aptitude  (50-65 
on  the  AQE) — on  troubleshooting  and  repair  tasks,  found  that  aptitude  had 
no  effect  on  errors  in  maintenance  repairs  or  on  the  time  required  for 
troubleshooting.  There  was  a statistically  significant  difference,  how- 
ever, in  errors  made  on  troubleshooting  at  both  the  system  module  and  defec- 
tive components  levels;  that  Is,  high-aptitude  perfoi^nce  was  better  than 
low-aptitude  performance. 


There  have  also  been  reports  of  successful  nontroubleshooting  per- 
formance by  personnel  considered  to  have  lower  aptitude  levels,  although 
the  studies  were  not  specifically  designed  to  test  the  aptitude  factors. 

Horn  (1972),  for  example,  found  lower-aptitude  mechanical  rat:).ngs  able  to 
do  electrical  ratings’  nontroubleshooting  jobs  with  FPJPAs.  Greater  per- 
formance from  iower-readlng-ablllty  personnel  may  also  be  Inferred  from  the 
successful  performance  of  maintenance  tasks  by  non- English  speaking  Vietnamese 
on  the  American  UH-IH  helicopter. 

Only  one  study  has  suggested  that  lower-aptitude  personnel  can 
also  do  troubleshooting  accurately  with  the  FPJPA:  Thelsen  and  Flshburne 
(lia  press).  In  a field-study  demonstration  with  41  Navy  technicians  whose 
selection  test  scores  were  below  the  cutoff  for  "A”  school  eligibility, 
concluded  that  the  subjects  were  able  to  satisfactorily  Isolate  and  repair 
malfunctions  in  the  AN-AQA-7  Sonar  Signal  Processing  System  with  the  use  of 
the  FPJPA.  Unfortunately,  the  results  of  this  study  were  greatly  weakened 
by  the  lack  of  adequate  control  data. 

The  consensus  of  researchers  In  the  field  Is  that  aptitude  trade- 
off has  not  been  adequately  studied.  However,  most  are  confident  that, 
whatever  the  level  of  performance  of  the  lower-aptitude  personnel.  Increased 
training  and  the  availability  of  JPAs  can  bring  even  very  low-aptitude 
personnel  (i.e.,  fifth-grade  reading  level  or  below)  to  acceptable  perfor- 
mance levels  In  non troubleshooting  operator  and  maintenance  tasks. 

Job  Experience 

Unlike  aptitude,  the  effects  of  experience  are  adequately  con- 
sidered in  most  JPA  experimental  research.  Rowan  (1973),  in  summarizing 
the  literature,  noted  that  all  FPJPA  experiments  and  field  tests  showed 
that  Inexperienced  technicians  performed  better  with  aiding  than  with  con- 
ventional documentation.  This  has  been  generally  true  for  troubleshooting 
as  well  as  nontroubleshooting  tasks,  although  acceptable  levels  on  complex 
troubleshooting  tasks  are  more  difficult  to  achieve  and  FPJPAs  have  failed 
completely  in  some  Instances  (Klesch,  1977). 

Some  JPA  techniquen  other  than  FPJPA  have  succeeded  In  raising  the 
performance  level  of  inexperienced  technicians  In  troubleshooting;  for 
example,  the  Augmented  Action  Tree  (Horn,  1973),  Logic  Tree  Troubleshooting 
Aids  (Potter  & Thomas,  1976),  and  the  Work  Package  Concept  (Horn,  Note  2). 

All,  however,  have  included  some  form  of  decision  logic  tree.  OriC  non- 
FPJPA  technique  that  has  received  considerable  attention  Is  the  MDC.  Studies 
of  the  use  of  this  technique  by  Inexperienced  personnel  have  yielded  mixed 
results  (Krohn,  1971;  Horn,  1973). 

Also,  although  all  experience  levels  generally  express  more  favor- 
able attitudes  toward  proceduraliaed  JPA  techniques  (Goff  et  al.,  1969; 

Potter  & Thomas,  1976),  well-prepared  conventional  documentation  In  the 
hands  of  experienced  personnel  can  result  in  better  performance  on  some 
types  of  troubleshooting  tasks  (Potter  & Thomas,  1976).  Highly  procedura- 
llzed  techniques  alone  probably  cannot  support  all  of  the  troubleshooting 
demands  of  complex  weapon  systems  (Klesch,  1977;  Booher,  1977b). 
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Type  of  Task 


For  determining  the  major  differential  effects  of  aiding  techniques, 
it  is  now  clear  that  the  type  of  maintenance  task  being  aided  can  be  clas- 
sified as  either  trotbleshootlng  or  nontroubleshooting. 

It  has  long  been  known  that  mechanics  can  become  relatively  skilled 
on  nontroubleshcotlng  tasks  after  6 months  of  field  experience,  whereas 
troubleshooting  skills  continue  to  improve  throughout  their  careers  (Baldwin, 
Mater,  Vineberg,  & Whipple,  1957).  With  job  aids,  inexperienced  and  lower- 
aptitude  personnel  can  be  proficient  on  nontroubleshooting  tasks  almost 
Immediately,  essentially  eliminating  the  first  6 months  of  warm-up.  Con- 
siderably more  time  is  required,  even  with  job  aids,  to  become  proficient 
in  troubleshooting  skills,  but  proficiency  can  be  achieved  much  earlier  in 
the  technician's  career  with  aiding. 

Procedurallzed  job  aids  are  less  expensive  to  develop  for  nontrouble- 
shooti7ig  tasks  than  for  troubleshooting  tasks  because  the  former  require  no 
functional  analysis  (Shrlver  & Hart,  1975)  and  are  much  easier  to  validate 
for  accuracy  (Johnson,  1977).  According  to  Middleton  et  al.  (1977),  non- 
troubleshooting  aids  are  also  relatively  easy  to  tailor  to  the  degree  of 
proceduralization  needed  to  meet  individual  maintenance  task  requirements. 

The  aiding  of  troubleshooting  tasks,  on  the  other  hand,  generally  requires 
a functional  analysis,  can  be  difficult  to  validate  for  accuracy,  and  inter- 
acts considerably  with  variables  such  as  technician  experience,  equipment 
complexity,  and  task  complexity. 

Complexity  of  Task  and  Equipment  Factors 

Studies  have  generally  attempted  to  deal  with  the  complexity  of 
tasks  and  with  the  type  and  complexity  of  equipment  by  sampling  across 
several  difficulty  levels  and  types  rather  than  considering  them  to  be 
major  Independent  variables.  Clear  differences  due  to  the  these  factors 
have  not  been  evident.  However,  some  general  rules  of  thumb  seem  to  have 
evolved  during  20  years  of  JPA  research.  First,  if  the  task  is  a nontrouble- 
shooting task,  then  task  complexity  and  the  type  and  complexity  of  equip- 
ment have  little  bearing  on  the  decision  concerning  the  type  of  JPA.  Second, 
some  types  of  tasks,  because  of  their  special  complexity  (e.g.,  difficult 
alignment,  adjustment  tasks),  might  require  additional  training.  Finally, 
if  personnel  of  varied  experience  are  to  use  the  aid,  then  a dual-level  pre- 
sentation of  procedures  may  be  useful.  Generally,  however,  these  three 
factors  do  not  create  a great  deal  of  difficulty  once  a JPA  approach  has 
been  selected.  The  guidelines  recommended  by  Middleton  et  al.  (1977)  appear 
quite  useful  as  format  selection  criteria  in  accordance  with  varying  task 
complexity  on  nontroubleshooting  tasks. 

If  the  task  is  a troubleshooting  task,  then  task  and  equipment  com- 
plexity, as  well  as  the  experience  and  training  background  of  the  personnel 
using  the  aid,  help  dictate  which  JPA  is  best.  Short  or  excessive  task 
length  can  greatly  influence  the  decision  to  use  either  the  HPJPA  or  a more 
deductive  form  of  aiding.  Other  subfactors  also  contribute  to  complexity, 
such  as  (1)  the  subordination  of  equipment  as  determined  by  the  top-down 
breakdown  (Post,  Price,  & Dlffley,  1976),  and  (2)  the  amount  of  special 
testing  and  unique  requirements  for  deductive  reasoning  (Middleton  et  al., 
1977). 
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FPJPA  approaches  are  generally  good  for  electrical,  mechanical,  or 
electro-mechanical  systems  (Post  & Brooks,  1970).  The  SIMM  or  FOMM  format, 
on  the  other  hand.  Is  usually  restricted  to  complex  systems;  It  Is  generally 
considered  better  for  analog  than  digital  electronics  and  not  good  at  all 
for  mechanical  systems  (Martin,  1975).  However,  no  aiding  systems  were 
found  that  could  be  considered  adequate  for  complex  digital  systems. 

Mechanical  systems  tend  to  be  aided  best  by  highly  pictorial  for- 
mats; at  least  that  Is  the  Inference  drawn  from  the  general  acceptance  of 
existing  pictorial  aids.  Haggard  and  Shock  (1962)  long  ago  demonstrated 
the  effectiveness  of  visual  aids  for  showing  signs  of  malfunction  and 
wear  In  equipment,  and  almost  all  mechanical  maintenance  personnel  use  the 
Illustrated  parts  breakdown  for  performing  maintenance  and  for  ordering 
replacement  parts.  Several  years  ago  the  Naval  Ordnance  Laboratory  and 
the  Army's  Human  Resources  Research  Organization  (HumRRO),  developed 
picture  guides  and  manuals  for  operators  and  malntalners  of  equipment 
such  as  tanks,  guns,  and  mines.  General  Motors  Corporation  (1977)  has 
developed  a fault  Isolation  and  repair  manual,  done  In  a highly  structured 
pictorial  format,  that  already  has  sold  more  than  250,000  copies. 

Procedurallzatlon 


In  any  model  for  JPA  selection,  probably  the  most  critical  factor 
regarding  effects  on  performance,  training,  or  job  design  is  the  degree  of 
procedurallzatlon.  One  of  the  earliest  studies  was  reported  by  Atchley 
and  Lehr  (1964).  Figures  A and  5 (Atchley  & Lehr,  1964)  show  the  time  and 
error  performance  advantages  possible  with  proceduralized  formats  as  com- 
pared to  other  types.  Job  aids  that  produced  the  performance  Improvements 
summarized  by  Rowan  (1973)  and  Shriver  and  Hart  (1975)  have  generally  relied 
heavily  on  procedurallzatlon,  as  determined  from  the  result  of  front-end 
analyses  of  equipment,  operator,  and  maintenance  task  requirements. 
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Figure  5.  Troubleshooting  performance  accuracy  of  Air  Force 
basic  trainees  with  three  JFA  format  components 


Media  and  Design  Factors 


In  selecting  the  media  and  physical  design  of  performance  aids,  two 
major  constraints  must  be  considered;  (1)  that  generally  Imposed  by  the 
hardware  system  Itself,  and  (2)  that  set  by  the  work  or  leanilng  environ- 
ment. 

Herd-copy  print  is  usually  recommended  because  (1)  it  is  the  least 
expensive  medium,  (2)  it  is  the  easiest  medium  to  use  on  the  job,  (3) 
performance  studies  have  shown  print  to  be  equal  to  or  &s  good  as  other 
media,  and  (4)  the  effect  of  JPAs  on  performance  is  content-dependent, 
not  medium-dependent.  There  may  be  situations,  however,  where  systems 
support  requirements  dictate  the  use  of  media  other  than  hard-copy  print; 
for  example,  there  could  be  distribution,  storage,  or  updating  problems 
with  the  print  format  chat  adversely  affect  the  aid's  timeliness  and 
accuracy.  Regardless  of  Che  medium,  any  aid  that  has  accurate,  complete 
data  is  more  essential  Co  job  performance  than  one  without  these  qualities. 
There  are  also  some  situations  where  the  task  may  not  be  presented  feasibly 
with  hard-copy  print.  For  example,  motion  pictures  could  be  used  to  describe 
complex  alignments  or  adjustments;  holography,  to  aid  three-dimensional 
judgments;  or  hand-held  computers,  to  facilitate  assisting  complex  mathema- 
tical operations.  In  addition,  costs  may  not  always  be  on  the  side  of  hard- 
copy print;  indeed,  paper  printing  costs  have  risen  exorbitantly  in  the  past 
few  years,  making  microform  more  and  more  appealing. 

Post,  Price,  and  Diffley  (1976)  said  that  three  primary  system  support 
requirements  influence  media  selection  that  is  based  on  varying  system 
design  and  operating  conditions;  (1)  access  to  information,  (2)  recording 
medium,  and  (3)  portrayal  mode.  The  first,  "access  to  information,"  is 
important  because  the  task  of  locating  information  often  consumes  20  per- 
cent of  the  maintenance  technician’s  time.  Important  man-system  Interface 
considerations  for  access  to  the  Information  depend  on  the  availability 
of  system  status  displays  and  automatic  test  equipment.  Patterns  of  dis- 
play readings  and  ATE  readouts  keyed  to  appropriate  job  aids  can  help  mini- 
mize the  Initial  search  time.  The  second  requirement,  "recording  medium," 
describes  such  things  as  system  size,  number  of  systems,  distribution  pro- 
cesses, and  installations  as  constraints  that  may  urge  choices  other  than 
hard-copy  print.  Post  et  al.  (1976)  recommended  that  paper  be  used  for 
information  bases  of  up  to  300  frames  while  the  microform  mode  is  more 
desirable  beyond  1500  frames.  The  third  requirement,  "portrayal  mode," 
is  most  critical  in  matching  work  place  conditions  to  job  aids  utilization. 
They  conclude  that  a single  medium  may  not  be  equally  suitable  for  all  system 
aids  or  even  for  all  aiding  applications  within  a system. 

Hughes  (1977b)  presented  a detailed  matrix  of  environmental  constraints 
between  and  within  systems  that  dictate  the  physical  design  of  hard-copy 
print,  microform,  and  audio  or  video.  An  ambitious  attempt  was  made  to 
recommend  design  choices  to  the  extent  of  page  size  and  typography.  The 
environmental  constraints  noted  are  generally  human  engineering  considera- 
tions and  include  work  space,  illumination,  dirt,  wind,  moisture,  temperature, 
noise,  time,  and  hazards  or  safety  restrictions.  Table  3,  a revision  of  the 
Hughes  matrix,  includes  several  additional  techniques.  For  a discussion  of 
the  relative  effectiveness  of  media  characteristics  developed  around  instruc- 
tional research,  see  Guilllford  (1973) . 
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Table  3 


£nvlronacntt^l^  ConiVrainta  '/at  Job/Tr«lttl)ig  Media 


Envli'ons.ant-aX  Conatraintii 


Vind/ 

Work  Koisture/ 

Medium  and  Characteristic)!  Space  Illumination  Dirt  Tcsperaturo  Motion  Mhlse  Tlistt  Keeard/Salcty 


Hard-copy  Prli'f 


Page  Siae  X 

Typography 

Pago  I.ayout  X 

Plcture/Word 
Cofflpoaltlon 
Color 

Page  Material  X 

Cover  Material  X 


Cover  and  Binding 
Craphlca 

Microform 

Portable  Viewer 


Hand-held  Viewer  X 

Printout  Capability  X 

Roll  Film  Cartier  X 

Cartridge/Caasette  X 

Aperture  Card 

Microfiche  X 

Frame  Retrieval  X 

Shaded  Screen  X 

Still  Plcturea 

Motion  Pictures  X 

Televlslon/Vldeo  X 

3-D  (Objects)  X 

3-D  (Holography)  X 

Audio  (Recording) 

Programgied  Instruction 

Text  X 

CAI  X 

Deuioiiutrallon  X 


X 


X 

X X 

X 


X 


X X 

X X 

X 
X 


X 

X 

X 

X X 


X 

X 

X 


X 

X 

X 


X 


X 

X 


X 

X 

X 


X 

X 

X X X. 

X 

X 

X 


X 


X X 


X 


- X 
X 
X 

X 


Oral  Presentation 


X X 


X 


Hand-Iield  Ccmputers 

Imagery  Enhancing 
Techniques 


X XX 

X 


X X 

X X 


Considerable  research  In  the  education  and  training  literature  shows  the 
advantages  of  various  media  in  wrk  environments  intended  to  meet  learning 
or  performance  objectives.  A modification  (Table  4)  of  Allen’s  (1967)  illus- 
tration of  the  relationship  of  instructional  media  to  learning  objectives 
is  considered  the  most  useful  classification  matrix  for  such  a comparison. 
Notice  that  no  media  type  can  be  ranked  "high"  in  meeting  the  objectives  of 
learning  factual  information,  performing  skilled  perceptual  motor  tasks,  or 
developing  desirable  attitudes. 

A final  factor  important  to  JPA  design  itself  is  the  amount  of  effort 
expended  on  improving  the  aid's  intelligibility  (Shriver  & Hart,  1975)  or 
comprehensibility  (Post  & Price,  1974).  This  includes  not  only  the  read- 
ability of  the  written  material  but  also  the  usability  of  graphics. 

Integrated  JPA/Training/Job  Design 

ITin  importance  or  integrating  the  JPA,  training,  and  job  design  cannot 
be  overemphasized.  The  three  have  been  considered  trade-offs,  depending 
on  cost,  performance,  and  manpower  limitations  in  meeting  overall  weapon 
systdm/miss.ion  objectives.  In  the  broadest  sense,  this  is  true.  The  avail- 
ablity  of  a highly  proceduralized  JPA  (l.e.,  an  FPJPA)  may  eliminate  the 
need  for  formal  training  and  allow  work  centers  to  better  design  the  work- 
load around  a complement  of  experienced  and  relatively  inexperienced  per- 
sonnel, but  the  VPJPA  would  probably  cost  more  than  a deductive  JPA,  Less 
expensive  deductive  aids,  on  the  other  hand,  require  greater  training  and 
on-the-job  experience  and  require  work  centers  to  operate  primarily  with 
highly  skilled  personnel. 

However,  this  seems  to  be  a gross  oversimplification  of  the  relationship 
among  these  three  technologies.  First,  no  one  technology  (i.e.,  JPA,  train- 
ing, or  job  design)  is  a direct  substitute  for  the  other.  All  three  must 
be  integrated  for  any  weapon  system.  The  savings  suggested  in  the  litera- 
ture for  military  training  (Shriver  & Hart,  1975)  as  a result  of  the  FPJPA 
cannot  be  realized  simply  by  reducing  training.  The  longer-term  personnel 
career  considerations  and  the  FPJPA 's  inadequacy  for  complex  troubleshooting 
tasks  must  also  be  taken  into  account  (Klesch,  1977). 

Second,  the  three  technologies  may  have  common  cost  areas.  Much  of  the 
cost  in  developing  a JPA  is  in  the  front-end  analysis  that,  to  a large  extent, 
is  fundamental  to  task-oriented  training  and  job  design.  Job  or  task  analyses 
designed  to  cover  the  requirements  of  all  three  technologies  during  weapon 
system  design  should  be  considerably  less  complex  and  costly  than  if  each 
area  conducted  independent  front-end  analyses. 

Further,  it  appears  increasingly  obvious  that,  after  major  manning 
decisions  have  been  made,  the  selection  of  a JPA  technology  should  be  made 
prior  to  the  selection  of  the  other  two  technologies.  Task-oriented  train- 
ing and  work  center  organization  can  be  more  readily  designed  to  support 
or  take  advantage  of  an  aid  than  if  the  aid  is  designed  to  complement  a 
style  of  training  or  work  center ' organization.  Also,  an  aid,  once  selected, 
is  more  similar  to  hardware  in  that  it  does  not  readily  adapt  to  individual 
differences,  whereas  training  and  job  design  should  by  their  very  nature 
strive  to  support  and  take  advantage  of  individual  differences.  The  problems 
of  trade-offs  among  the  three  disciplines  is  really  one  of  systematically 
considering  each  of  the  technology  areas  at  the  appropriate  stages  of  weapon 
system  RDT&E, 
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Relative  Values  of  Various  Media  for  Meeting 
Training/Performance  Objectives 


Adapted  from  Allen  (1967). 


r 
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Training  Interface  with  JPA 

The  earlier  studies  done  for  the  Army  by  HumRRO  (Shriver,  1960; 

Rogers  & Thorne,  1965;  Gebhard,  1970)  provide  the  best  data  for  integrated 
job  aiding  and  training  between  the  two  extremes  of  FPJPA  with  minimal  or 
no  training  and  conventional  technical  manual  aiding  with  extensive  formal 
training.  On  the  M-33  antiaircraft  fire  control  system,  for  example,  the 
Army  found  that  electronics  technicians  using  FORECAST  aids  could  perform 
as  well  as  conventionally  trained  technicians  even  though  their  training 
was  reduced  from  30  weeks  to  12  weeks  (Shriver,  1960).  Similarly,  Army  radio 
relay  and  carrier  repair  technicians  using  JOBTRAIN  aids  with  a 50  percent 
reduction  in  training  time  were  able  to  perform  as  well  as  conventionally 
trained  technicians  using  conventional  documentation  (Gebhard,  1970). 

A common  question  regarding  the  relationship  of  training  and  aiding 
asks  what  information  is  best  presented  via  each  technology.  The  literature 
provides  numerous  guidelines  for  assigning  performance  task  coverage  (Chalupsky 
& Kopf,  1967;  Chenzoff,  1973;  Llneberry,  1977).  Information  about  the  fol- 
lowing types  of  tasks  would  best  be  presented  via  a training  technology: 

1.  Tasks  that  are  not  very  easy  to  learn  on  the  job. 

2.  Tasks  that  are  hard  to  communicate  with  words. 

3.  Tasks  that  need  a great  deal  of  practice  for  acceptable  per- 
formance to  be  established. 

4.  Tasks  that  allow  little  room  fo’-  error. 

5.  Tasks  with  serious  consequences  for  errors. 

6.  Tasks  with  reasonable  training  costs. 

7.  Tasks  that  are  performed  frequently  on  the  job. 

8.  Tasks  for  which  the  required  speed  or  response  rate  does 
not  permit  referral  to  a manual. 

9.  Tasks  performed  by  a large  proportion  of  the  individuals 
in  a given  specialty. 

10.  Tasks  that  can  be  performed  with  considerable  freedom  in 
individual  style. 

11.  Tasks  that  are  so  new  or  undefined  that  the  detailed  steps  of 
ir.astary  are  unknown. 

12.  Tasks  wherein  psychosocial  factors  are  important. 

An  aiding  technology  would  be  the  best  method  of  presenting  informa- 
tion about  the  following  ♦•ypes  of  tasks: 

1.  Tasks  that  are  usually  performed  in  a fixed  sequence. 


I 
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2.  Tasks  that  involve  long  or  complex  behavior  sequences. 
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3.  Tasks  that  are  performed  Infrequently. 

4.  Tasks  that  involve  readings  and  tolerances. 

5.  Tasks  that  can  be  rehearsed  mentally  before  the  need  to 
perform  them  arises. 

6.  Tasks  that  are  aided  by  illustrations. 

7.  Tasks  that  utilize  reference  information  such  as  tables, 
graphs,  flow  charts,  and  schematics. 

8.  Tasks  with  branching  step  structures. 

9.  Tasks  remaining  stable  over  long  periods  of  time. 

10.  Tasks  for  which  established  task  sequences  are  critical. 

11.  Tasks  for  which  the  performance  method  is  likely  to  change. 

12.  Tasks  that  must  be  treated  with  a very  limited  training 

budget. 


As  discussed  in  the  previous  section,  a distinction  is  sometimes 
made  between  tasks  that  need  to  be  memorized  (head)  and  those  that  can  be 
aided  (book).  Maintaining  a separation  between  maintenance  training  and 
aiding  tasks,  when  the  aid  is  an  FPJPA  and  the  training  is  "hands-on  experi- 
ence," becomes  extremely  difficult.  However,  the  difference  between  what 
is  important  to  know  without  an  aid  and  what  is  important  to  be  able  to 
do  with  an  aid  is  still  a good  conceptual  distinction. 

Job  Design  Interface  with  JPA 

Perhaps  even  more  important  than  JPAs’  effects  on  training  are  their 
effects  on  job  design.  Attempts  to  design  a job  or  work  center  organiza- 
tion to  take  advantage  of  performance  aiding  have  been  few,  although  the 
potential  manpower  benefits  are  great. 

Post  and  Brooks  (1970)  developed  a mathematical  model  of  the  F-4J 
work  center  and  applied  it  to  the  data  collected  under  NAVAIR's  AMSAS  experi- 
mental study  program.  The  findings  demonstrated  job  guides’  ability  to  aid 
performance  on  nontroubleshooting  tasks  for  electrical,  mechanical,  and 
electromechanical  equipments.  In  that  study,  71  percent  of  the  inexperienced 
technicians'  time  was  spent  observing  and  assisting  experienced  technicians, 
and  the  other  29  percent  could  not  be  assigned.  If  job  guides  were  used, 
then  83  percent  of  inexperienced  labor's  time  could  be  spent  on  maintenance. 
The  provision  of  job  aids,  therefore,  could  result  in  major  job  design  alter- 
natives and  flexibility  for  a work  center.  Post  and  Brooks'  findings  that 
a work  center  could  expect  a 52  percent  increase  in  the  availability  of  ex- 
perienced technicians,  a 25  percent  decrease  in  maintenance  queues,  and  a 75 
percent  decrease  in  failed  maintenance  actions,  show  that  it  would  actually 
be  feasible  to  Increase  operational  readiness  while  reducing  the  number  of 
maintenance  personnel. 
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Horn  (Note  2)  demonstrated  in  field  studies  with  FPJPA  and  Work 
Packages  that  a ratio  of  one  experienced  to  four  inexperienced  technicians 
can  perform  all  the  maintenance  actions  of  a work  center. 

JPA/Training/Job  Design  Interfaces 

Several  Important  factors  must  be  considered  in  job  design  inter- 
faces with  JPA  and  training.  The  following  factors  have  received  the 
most  attention:  task  organization,  position  definition,  task  modules  or 
clusters,  skill  transfer  areas,  graded  proficiency  levels,  span  of  super- 
vison,  teaming,  spares  philosophy,  equipment  test  philosophy,  and  the  use 
of  ATE. 


Skill  transfer,  for  example,  is  always  an  Important  consideration 
in  military  operations.  The  ability  to  use  personneJ,  in  operator  or  main- 
tenance tasks  for  which  they  have  not  been  formally  trained  is  especially 
Important  in  combat  or  other  emergencies.  Enlarging  the  scope  of  a job  for 
career  advancement  or  to  balance  workloads  on  a day-to-day  basis  is  also 
more  feasible  if  personnel  can  move  readily  from  one  job  to  another.  Job 
aids’  potential  for  facilitating  such  transfers  has  been  demonstrated  on 
several  occasions.  Rigney,  Fromer,  Langston,  and  Adams  (1965a,  1965b) 
found  that,  with  a fault  locator  aid  (XFL),  radio  operators  with  little 
or  no  experience  in  electronics  could  readily  solve  radio  problems.  Horn 
(1972)  also  found  that  electronics  and  mechanical  ratings  could  satisfac- 
torily perform  each  other’s  jobs  on  a mobile  electric  power  plant  with 
the  help  of  FPJPAs. 

There  has  been  some  work  to  improve  the  methodology  and  provide 
guidance  for  determining  an  optimum  JPA/ training/ job  design  interface. 

In  the  model  proposed  by  Post,  Price,  and  Dlffley  (1976)  and  Post  (i977a) 
for  selecting  formats  and  media  to  present  maintenance  information,  work 
center  considerations  such  as  span  of  supervision,  personnel  turnover,  and 
maintenance  demands  (workload  over  time)  were  considered  key  decision  criteria. 
It  is  unlikely,  however,  that  data  on  those  variables  will  be  available  early 
enough  to  make  such  sensitive  selections  on  new  system  procurements.  Here 
again,  it  appears  that  the  decision  for  a particular  type  of  job  aid  may 
be  predetermined  because  of  the  personnel  and  training  cost  savings. 

Guidelines  for  position  (or  job  design)  and  documentation  interface 
have  also  been  developed  for  the  Bell  Laboratories  (Price,  Post,  & Barker, 

1973;  Post,  1977b).  These  should  be  useful  in  the  development  of  military 
systems  to  integrate  JFAs,  training,  and  job  design  for  operation  and  main- 
tenance tasks.  The  following  is  a list  of  job  design  factors  that  have  been 
shown  to  be  important  to  work  productivity  (Post,  1977b): 
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Factors 


Design  Guidelines 


Process 

1.  Sequence 

2.  Dependencies 

3.  Functional  Integrity 

4.  Workload 

5.  Difficulty 

Personnel 

6.  Complete  Work 

7.  Autonomy 

8.  Comprehension 

9.  Challenge 

10.  Valued  Abilities 

11.  Meaningfulness 

12.  Participation 

( 


Sequential  tasks  are  good  candidates 
to  Include  In  a job. 

Dependent  tasks,  even  though  they 
might  not  be  sequential,  are  good 
candidates  to  Include  In  a Job. 

Tasks  with  a common  objective  are 
good  candidates  to  Include  In  a job. 

The  timing  and  duration  of  job  tasks 
should  be  considered  so  as  not  to  over- 
or  underload  the  Incumbents. 

Task  difficulty  should  be  compatible 
with  Incumbents'  performance  capa- 
bilities and  should  be  relatively 
homogeneous  across  the  job. 


Job  tasks  should  result  in  a sense 
of  completion  or  closure. 

Job  tasks  should  permit  the  Incumbent 
to  plan  and  control  his  own  work. 

Job  tasks  should  be  understood  (both 
how  and  why)  to  promote  performance 
reliability. 

Job  tasks  should  be  challenging,  yet 
within  the  Incumbent's  capability. 

Job  tasks  should  require  significant 
abilities  (skills/knowledge). 

The  product  of  job  tasks  should  be 
worthwhile  and  have  a personalized 
nature  (e.g,,  "my  equipment"). 

An  Incumbent  should  be  able  to  relate 
his  tasks  to  the  organization's  goal 
or  product. 

Job  tasks  should  provide  direct  or 
Indirect  opportunities  to  learn  new 
and  valued  skills. 


JPA  Test  and  Evaluation  Technology 


Without  a method  for  assessing  a JPA's  value  relative  to  conpet^Ag  tech- 
nologies and  to  other  JPAs,  there  Is  no  basis  for  knowing  which  JPA  would 
serve  best  or  whether  or  not  a JPA  should  be  used  at  all.  Tools  for  making 
such  an  assessment  generally  can  be  classified  as  survey,  experimental 
(laboratory),  demonstration  (field),  and  simulation. 

Simulation 


JPA  technology  has  been  tested  and  evaluated  extensively  with  all 
four  techniques.  Losee  et  al.  (1962)  and  Main  (1974)  are  good  exanq>les  of 
using  the  survey  technique.  Whether  a survey  employs  a questionnaire, 
an  Interview,  or  some  condilnatlon  of  the  two,  it  can  obtain  a large  amount 
of  data  relatively  quickly  and  Inexpensively.  The  survey  can  be  used 
(1)  to  determine  whether  users  would  prefer  a hi.ghly  procedurallzed  format 
In  pocket  book  form  to  a deductive  format  with  all  the  Information  on  one 
page,  (2)  to  guide  the  choice  between  a printed  page  or  microform  frame 
display  at  the  user  site,  or  (3)  to  save  the  cost  of  printing  expensive 
color-coded  troubleshooting  charts  when  other  methods  of  coding  would  be 
just  as  effective. 

The  preferred  method  of  collecting  data  for  use  In  decision  making 
at  all  levels  Is  the  controlled  experiment,  either  in  the  laboratory  or  in 
the  field.  However,  of  all  the  JPA  systems  and  techniques  listed  in  the 
glossary,  less  than  25  percent  have  laboratory  or  field  performance  data 
to  demonstrate  their  effectiveness.  Most  of  these,  as  Rowan  (1973)  pointed 
out,  simply  compared  the  techniques  against  "traditional"  documentation. 

Only  a handful  of  studies  have  compared  more  than  two  techniques  or  have 
attempted  to  Improve  the  conventional  technique  before  testing.  Studies 
that  have  attempted  to  Improve  the  accuracy,  completeness,  compactness, 
and  readability  of  the  conventional  technique  Include  Post  and  Brooks  (1970) 
on  nontroubleshooting  tasks  and  Elliott  and  Joyce  (1968)  on  troubleshooting 
tasks.  Studies  that  have  compared  more  than  two  techniques  Include  Atchley 
and  Lehr  (1964)  on  procedurallzed  Instructions  versus  data  flow  versus 
schematics;  Horn  (1973),  on  Augmented  Action  Tree  (AAT)  versus  MDC  versus 
conventional  Navy  technical  manual;  Potter  and  Thomas  (1976),  on  FPJPA  versus 
Logic  Tree  versus  Air  Force  Technical  Order;  Booher  (1975),  on  six  types 
of  procedurallzed  instructions;  and  Frey  (1976),  on  three  types  of  pictorial 
media. 


Because  of  the  difficulty  and  expense  of  collecting  data  experi- 
mentally, computer  simulation  models  are  often  considered  as  an  alternative. 
A typical  demonstration  In  the  field  may  cost  several  hundred  thousand 
dollars,  the  bulk  of  which  goes  for  the  preparation  of  JPA  test  materials. 
Even  a controlled  laboratory  experiment  that  deals  with  a few  tasks  or  sub- 
tasks may  easily  cost  $50,000  to  $100,000.  However,  the  data  used  to 
validate  simulation  models,  even  In  fields  other  than  JPA,  are  often  so 
unreliable  that  simulation  Is  seldom  applied. 

The  best  applications  of  test  and  evaluation  models  have  been  system 
effectiveness  estimates  using  simpler  mathematical  models.  Post  and  Brooks' 
(1970)  effort  on  manpower  utilization  is  a good  example.  The  only  other 
study  of  this  nature  concerned  the  Air  Force  C-141A  aircraft.  The  analyses 
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reported  in  the  PIMO  final  report  (Serendipity,  1969)  and  by  Braid  (1975) 
indicate  that  simply  reallocating  time  from  OJT  to  maintenance  would  de- 
crease the  total  time  a one-term  enlistee  spends  in  training  by  25  per- 
cent. Other  predictions  were  that  flight  departure  reliability  could  be 
improved  by  50  to  65  percent;  operational  readiness  time,  by  38  to  40  per- 
cent; unscheduled  maintenance  manpower,  by  30  to  39  percent;  and  spares 
demands,  by  30  percent.  If  these  results — assuming  that  the  conservative 
estimates  made  by  Rowan  (1973),  Shriver  and  Hart  (1975),  and  Shriver  (1977) 
are  valid — could  be  fully  applied  to  Navy  personnel,  training,  and  main- 
tenance support  associated  with  electronic  systems,  then  the  Navy  could 
expect  (1)  decreases  in  first-enlistment  training  costs  of  50  to  80  per- 
cent, or  $900  million  annually,  (2)  savings  in  spares  of  $75  million  per 
year  due  to  reductions  in  erroneous  removal,  and  (3)  a decrease  in  main- 
tenance labor  force  of  15  percent,  or  $700  million  annually. 

Other  payoffs  are  projected,  but  they  have  no  quantitative  basis 
for  evaluation:  (1)  an  Increase  in  the  effective  use  of  lower-skilled  per- 
sonnel, (2)  improved  job  Involvement  and  satisfaction  mnong  first-term  en- 
listees, and  (3)  an  Increase  in  retention  rates  for  productive  personnel. 

One  simulation  technique  for  assessing  and  predicting  the  relative 
performance  effectiveness  of  job  performance  aids  was  developed  at  North 
Carolina  State  University  (Smillie,  Edsall,  Ayoub,  & Muller,  1976),  That 
approach  combines  features  of  predetermined  time  and  motion  systems  (PTMS); 
a simulation  modeling  algorithm,  Graphical  Evaluation  and  Review  Techniques 
(GERT) ; and  the'  Systems  Analysis  of  Integrated  Networks  of  Tasks  (SAINT) . 

Results  of  the  simulation  process  should  yield  substantial  informa- 
tion on  JPAs*  effects  on  performance  time  and  errors.  For  Instance,  statistics 
can  be  obtained  on  the  total  performance  time  and  the  probability  of  success- 
fully completing  the  task  while  using  a specific  JPA.  In  addition,  simula- 
tion results  can  be  used  to  identify  the  task  step  or  steps  that  might  be 
crucial  to  the  success  or  failure  of  the  task  at  hand.  Based  on  these 
statistics  and  the  cost  data  associated  with  JPA  production  and  maintenance, 
a cost/benefit  analysis  can  be  performed  to  determine  the  best  JPA-maintenance 
system  or  design — a design  that  could  be  expected  to  yield  minimum  performance 
time  and  errors  at  minimum  cost. 

Unfortunately,  the  model  is  not  in  a form  that  can  be  readily  applied 
to  specific  JPAs,  training,  or  job  designs.  It  appears  to  be  a rather  com- 
plicated way  of  analyzing  relatively  simple  task  steps,  and  the  model  was 
never  validated.  Model  development  and  validation  were  attempted  on  such 
simple  tasks  that  time  and  error  differences  obtained  in  both  the  experi- 
ment and  simulation  were  insignificant.  Attenpts  to  further  reduce  task 
steps  into  specific  behavioral  processes  would  not  be  meaningful  on  such 
easy  operations.  Furthermore,  no  attempt  was  made  to  address  troubleshooting 
tasks,  where  it  now  appaars  that  the  greatest  difficulty  for  JPA  selection 
lies.  At  present,  therefore,  no  usable  mathematical  model  or  simulation 
technique  exists  for  assessing  or  predicting  the  effectiveness  of  job  per- 
formance aids. 


CONCEPTUAL  ORGANIZATION  OF  JPA  TECHNOLOGY 


Development/ Selection  Methodology 

The  previous  two  sections  of  this  report  described  the  research  and  tech- 
niques that  comprise  the  current  JPA  technology  base.  Unfortunately,  the  way 
JPA  techniques  have  evolved  and  research  has  been  conducted  has  not  made  the 
most  efficient  use  of  available  resources.  Also,  most  of  the  work  was  done 
without  thought  to  fitting  it  into  a larger  conceptual  framework  that  would 
later  be  useful  to  researchers,  developers,  and  implementers. 

The  development  or  selection  of  JPA  techniques  has  usually  been  done 
by  one  of  five  methods: 

1.  Trial  and  error.  With  this  technique,  which  is  the  most  common, 

an  existing  aid  form  (most  often  a technical  manual  for  equipment  operation 
or  maintenance)  is  gradually  changed  over  time  to  meet  user  demands.  Re- 
search data  are  seldom  collected.  General  user  acceptance  is  co  >3idered 
the  primary  measure  of  success.  Although  this  approach  is  gener-xly  quite 
inefficient,  it  has  yielded  some  very  good  JPAs  (e.g.,  the  NAVAIR  Work 
Package  concept,  the  General  Motors  Diagnosis  and  Repair  Manual). 

2.  Fiat.  Although  commonly  applied  in  the  military  services  to  procure 
JPAs,  this  is  the  worst  method.  Under  it,  a system  is  built  and  lnq>lemented 
by  decree.  This  method  is  largely  to  blame  for  the  problems  of  the  NAVAIR 
and  NAVSEA  microform  systems. 

3.  Brochuremanshlp . Each  of  the  101  performance  aids  listed  in  the 
glossary  has  probably,  at  one  time  or  another,  been  sold  to  a project  manager 
simply  on  the  basis  of  advertised  advantages  over  the  "conventional  technique. 
This  is  known  as  brochuremanshlp. 

4.  Literature  extrapolation.  A few  performance  aids  (e.g.,  the  FPJPA 
type  shown  in  Figure  1)  have  been  developed  by  extrapolation  from  the  litera- 
ture. These  Incorporate  such  features  as  short  sentences,  short  words,  verb 
lists  from  readability  research,  and  the  limitation  of  seven  callouts  per 
page.  This  method  provides  a good  quality  system  but  may  result  in  higher 
costs  than  necessary. 

5.  Algorithm.  The  technique  stimulating  the  greatest  Interest  recently 
has  been  that  of  building  and  selecting  the  optimum  aid  form,  considering 
(a)  the  combined  effects  of  several  variables  on  user  performance  and  (b) 
the  cost  factors.  There  have  been  a number  of  attempts  to  develop  models 

or  algorithms  for  this  purpose,  but  no  aid  forms  have  actually  been  con- 
structed using  these  techniques.  Inherent  in  this  approach  is  the  assump- 
tion that  selecting  the  optimum  JPA  depends  on  simultaneous  consideration  of 
major  variables  such  as  environment,  personnel  characteristics,  and  type  of 
task,  much  as  diagramed  in  Figure  6,  a conceptual  model  for  JPA  technology 
utilization  (Blanchard,  1977). 


None  of  the  techniques,  including  the  algorithm,  has  provided  a satis- 
factory way  of  organizing  JPA  technology  for  research  or  implementation 
decisions.  For  that  reason,  an  attempt  was  been  made  to  formalize  an 
organization  of  JPA  technology  that  might  be  conceptually  useful  to  those 
employed  In  either  applying  or  advancing  the  current  technology.  Hope- 
fully, the  concept  presented  here  will  help  organize  the  available  tech- 
nology for  easier  application  decisions  and  for  better  determination  of 
which  future  research  should  be  emphasized. 

Organizing  the  Model 

A simple  model  for  organizing  JPA  technology  was  found  useful  for  re- 
viewing the  techniques  listed  In  the  glossary,  for  appraising  the  various 
algorithms  recently  developed  in  other  military  programs,  and  for  Identify- 
ing the  known  behavioral  concepts  applicable  to  JPA  basic  research.  Briefly, 
as  illustrated  in  Figure  7,  this  model  consists  of  three  major  JPA  tech- 
nology levels,  each  of  which  has  a number  of  distinguishing  characteristics. 

The  levels  are  (1)  the  JPA  system  level,  (2)  the  performance  aiding  component 
level,  and  (3)'  the  performance  aiding  element  level.  Depending  on  which 
level  Is  addressed,  there  can  be  considerable  variation  In  the  performance 
aid  structural  form,  research  questions,  application  questions,  and  types  of 
expertise  involved. 

Two  Important  considerations  become  Immediately  obvious  from  this  organiza- 
tion. First,  Figure  6*s  three-dimensional  arrangement  of  variables  Is  use- 
ful only  after  one  decides  what  level  Is  being  addressed.  Applying  variables 
directly  to  JPA  systems,  components,  and  elements  alike  without  Identifying 
what  part  of  the  larger  system  Is  being  described  simply  adds  to  the  exist- 
ing confusion.  Second,  the  organizing  model  provides  a visual  reminder  of 
the  major  differences  between  the  optimum  paths  for  (1)  the  design  of  JPAs 
and  (2)  for  the  application  of  JPAs  in  personnel  systems.  The  design  build- 
ing path  should  move  from  the  element  level  to  the  system  level,  whereas 
the  critical  factors  In  JPA  application  diminish  In  Importance  as  one  moves 
from  the  system  level  to  the  element  level.  The  structure  of  the  three 
different  levels  is  detailed  In  Table  5. 


JPA  Systems  Level 


A substantial  body  of  literature,  describes  two>way  comparisons 
between  new  aiding  concepts  and  conventional  practice.  Summaries  (Rowan, 

1973;  Shrlver  & Hart,  1975)  almost  always  show  that  aiding  enhances  on-the- 
job  performance.  Some  job  aids  simply  allow  skilled  people  to  do  better, 
some  allow  unskilled  people  to  do  jobs  they  couldn't  otherwise  do,  and  some 
allow  jobs  to  be  done  better  or  the  same  with  fever  people.  The  selection 
of  one  of  these  types  depends  on  factors  that  must  be  treated  at  fairly 
gross  levels  (e.g.,  troubleshooting  tasks  versus  nontroubleshooting;  low. 
Intermediate,  or  high  technician  aptitude;  high  versus  low  equipment  com- 
plexity). Data  at  this  level  are  primarily  useful  for  JPA  systems  decisions. 
An  attempt  Is  generally  made  to  provide  JPAs  that  cover  all  or  large  portions 
of  several  task  groups  required  to  support  a particular  hardware  system  or 
equipment.  It  therefore  appears  that  the  term  "job"  Is  most  appropriate  at 
this  level.  Implementation  usually  requires  one  specification  for  development, 
test,  and  evaluation,  and  another  for  associated  training  requirements. 

Good  examples  of  JPA  systems  are  the  Air  Force's  Fully  Procedurallzed  Job 
Performance  Aid  ^FPJPA),  the  Army's  Improved  Technical  Data  and  Training 
(ITDT),  and  the  Navy's  Functionally  Oriented  Maintenance  Manual  (FOMH). 

JPA  Component  Level 

The  JPA  system  Is  generally  comprised  of  several  different  types  of 
presentation  styles  or  formats,  each  designed  to  aid  a different  category 
of  behavioral  tasks  (e.g.,  lubricate,  remove,  fault  isolate,  test).  The 
FOMM,  for  example,  contains  tables,  lists,  functional  blocks,  matrices, 
written  procedures,  pictures,  and  schematics,  each  designed  to  aid  different 
aspects  of  the  maintenance  tasks  of  any  particular  equipment.  These  perfor- 
mance aids,  when  used  to  present  information,  are  In  forms  that  tend  to 
occupy  the  user's  total  field  of  view  at  any  one  time.  Commonly  described 
as  pages,  frames,  or  display  events,  such  units  exemplify  the  component  level 
of  JPA  technology. 

JFA  Element  Level 


In  many  Instances,  the  presentation  component  is  used  to  aid  a large 
number  of  different  behavioral  subtasks.  Consider,  for  example,  components 
such  as  an  Illustrated  parts  breakdown  diagram  or  an  electronic  schematic. 
Specific  behavioral  subtasks  (e.g.,  using  callouts,  searching  for  functional 
areas,  following  electron  flow,  reading  voltages,  reading  wave  forms)  may 
be  aided  by  different  features  of  the  same  presentation  component  being  used. 
Many  times,  features  that  show  positive  performance  advantages  by  themselves 
Interact  negatively  with  positive  performance  features  of  other  aiding 
techniques  at  this  level.  A good  example  of  this  Is  the  attempt  to  put 
on  one  page  a large  number  of  callouts  to  a figure.  There  can  be  good  per- 
formance advantages  In  eliminating  references  to  other  pages,  but  performance 
disadvantages  will  soon  arise  due  to  perceptual  difficulties  In  Isolating 
the  correct  callout.  This  level,  the  foundation  for  the  component  and  system 
levels.  Is  concerned  with  organizing  information  within  a page,  slide,  film 
strip,  etc.  Investigation  at  this  level  most  closely  resembles  conventional 
psychological  experimental  research  In  a controlled  environment.  This  Is 
also  the  level  where  the  greatest  amount  of  extrapolation  from  the  experimental 
literature  has  been  attempted.  In  new  -JPA  development,  caution  should  be 
exercised  because  laboratory  data  have  seldom  been  collected  on  tasks  that 
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resemble  the  actual  operator-malntalner  task.  (For  example,  results  from 
research  on  perceptual  recognition,  reading  speed,  and  reading  comprehension 
are  almost  useless  for  application  to  the  understandablllty  of  short,  pro- 
cedurallzed  Instructions.)  Furthermore,  almost  no  data  deal  with  the 
numerous  Interactions  among  the  behavioral  subtasks  Involved  In  using  any 
performance  aid  presentation  element. 

Decision  Making  at  the  JPA  Levels 

Table  5 presents  some  examples  to  Illustrate  decision  making  at 
the  three  JPA  technology  levels.  The  different  objectives  and  anticipated 
results  depend  greatly  on  which  level  Is  being  addressed.  In  a long-term 
research  program,  the  general  direction  In  the  development  of  a technique 
Is  from  right  to  left  In  the  table.  In  an  Implementation  program,  however, 
the  criticality  of  the  decisions  In  terms  of  cost/performance  trade-offs 
and  critical  time  for  decision  making  Is  from  left  to  right. 

Application  questions  at  the  system  level  might  be,  "Are  a task 
analysis  or  special  training  requirements  necessary?"  or  "What  manpower  limi- 
tations should  be  considered?"  At  the  component  level,  the  questions  revolve 
around  formatting  interface,  content  coverage,  and  requirements  for  special 
information  storage  and  retrieval  equipment.  Research  objectives  vary  from 
identifying  trade-offs  among  major  technology  areas  at  the  system  level  to 
determining  principles  of  technical  material  readability  at  the  presentation 
element  level. 

Perhaps  one  of  the  most  useful  applications  of  this  concept  Is  In 
Identifying  the  decision  makers  who  play  major  roles  at  the  different  levels. 
Top  management  decides  when  and  where  to  use  the  overall  JPA  system,  while 
personnel  and  training  organizations  decide  which  career  paths  can  be  made 
available  with  any  particular  aiding  concept.  However,  top  management  should 
allow  flexibility  for  the  managers  of  specific  programs  to  select  the  best 
combinations  of  schematics.  Illustrations,  and  special  media  to  meet  unique 
mission  and  environmental  requirements.  On  the  other  hand,  decisions  already 
made  at  the  system  level  and  principles  developed  at  the  element  level  will 
limit  the  choices  to  be  made  at  the  component  level.  For  example,  if  the 
system  selection  Is  FPJPA  for  personnel  and  training  reasons,  then  the  com- 
ponent choices  are  pretty  well  limited  to  illustrations,  logic  trees,  and 
written  procedures.  Therefore,  the  program  manager  decision-making  level 
that  most  JPA  selection  algorithms  have  been  attempting  to  address  may  not 
have  as  critical  a need  for  selection  criteria  as  the  other  two  levels. 

JPA  Organizing  Concept  Applications 

There  are  at  least  three  Important  application  areas  where  the  JPA 
organizing  concept  appears  useful  for  grouping  the  theoretical  and  experi- 
mental literature;  (1)  cost  trade-off  analyses  among  performance  aids,  (2) 

JPA  selection  algorithms,  and  (3)  theoretical  trends  In  performance  aiding. 

Cost  Trade-off  Analyses 

One  of  the  first  applications  that  can  be  made  with  the  JPA  organizing 
concept  Is  In  grouping  the  theoretical  and  experimental  literature  for  cost 
trade-off  analysis.  At  the  systems  levels,  decision  makers  should  look  for 
data  that  support  major  trade-offs  among  technologies  as  well  as  among 
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aiding  systems.  The  data  on  the  FPJFA  system  strongly  Indicate  that  little 
or  no  preassignment  training  Is  required  If  the  highly  procedurallzed  aid 
form  can  be  used.  Less  procedurallzed  systems,  such  as  the  Army's  FORECAST 
(Shrlver  & Trexler,  1965)  and  JOBTRAIN  (Gebhard,  1970),  show  a 50  percent 
reduction  In  training  with  Joint  aiding  and  training  development.  One  field 
researcher  (Korn,  Note  2)  obtained  data  that  suggest  that  major  Job  design 
alternatives  are  possible  with  procedurallzed  aids.  Horn's  results  showed 
that  If  the  organization  permits,  one  skilled  technician  can  keep  four 
inexperienced  technicians  busy  through  the  use  of  Job  aids.  One  of  the 
few  comparisons  that . Included  more  than  one  wTPA  system  alternative  was 
recently  completed  by  Potter  and  Thomas  (1976).  Table  6 shows  that  LTTA, 
which  is  not  as  fully  procedurallzed  as  the  PPTA  but  Is  considerably  cheaper, 
shows  performance  advantages  over  conventional  techniques  very  similar 
to  the  advantages  of  the  FPTA.  A cost/performance  trade-off  analysis 
from  these  data  could  easily  lead  to  an  organizational  decision  In  favor 
of  the  LTTA  over  the  FPTA  as  the  optimum  aiding  system. 


Table  6 

Comparison  of  Three  JPA  Troubleshooting  Systems 
for  Performance  Accuracy,  Parts  Consumption,  and  Speed 


System 

Accuracy* 

Parts  Consumption^ 

Speed® 

FPTA 

95.4 

.72 

30.8 

LTTA 

89.4 

1.05 

34.8 

TO 

60.6 

2.67 

59.9 

Note.  From  Potter  and  Thomas  (1976). 
®Mean  proportion  of  problems  solved. 
^Mean  number  of  parts  used  Incorrectly. 
‘^Mean  time  In  minutes. 


There  are  few  good  studies  that  give  data  on  component-level  compari- 
sons. One  notable  exception  Is  Atchley  and  Lehr  (1964).  Their  data,  shown 
previously  In  Figures  4 and  5,  fit  well  at  the  component  level  because  coiiq>srl- 
sons  are  made  of  several  distinct  presentation  components  and  varying  levels 
of  complexity.  These  show  clearly  the  advantages  of  procedurallzed  instruc- 
tion over  data  flow  diagrams  and  standard  schematics.  Unfortunately,  this 
technique  Is  also  the  most  expensive  of  the  three. 
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Table  7 sunanarlzes  a comparison  of  plcture/word  combination  formats 
(Booher,  1975)  that  might  fit  best  at  the  element  level.  Several  subtasks 
were  measured  for  performance  time  and  errors  In  looking  for  the  optimum 
organization  of  pictures  and  words  within  a component  (in  this  case,  a pro- 
ceduralized  Instruction).  The  data  clearly  indicate  that  Format  IV,  the 
pictorial/related  word  format  (see  Figure  8 for  an  example),  is  optimum  for 
both  performance  measures;  however.  It  may  not  be  the  most  cost-effective 
(Booher,  1975).  Formats  III  and  IV  (the  pictorial  formats)  are  about  equal 
in  price,  at  least  double  the  cost  of  Formats  I and  II  (the  verbal  formats). 
Format  III,  the  completely  pictorial  format,  can  be  eliminated  on  performance 
factors,  leaving  I,  II,  and  IV.  Surprisingly,  Format  I (with  no  pictures) 
may  be  adequate  if  time  is  not  a major  consideration,  whereas  Format  II 
(Figure  9,  from  Booher,  1972),  the  one  most  like  the  best  FPJPA  and  PIMO 
formats,  would  not  come  out  particularly  well  in  a trade-off  analysis  be- 
cause there  are  few  Instances  in  which  it  is  desirable  to  sacrifice  accuracy 
for  speed. 


Table  7 ! 

Speed  and  Accuracy  of  Navy  Maintenance  Personnel  Completing  a Series  of 
Behavioral  Subtasks  Using  ProcedurallzeJ  Instructions 
with  Varying  Plcture/Word  Combinations 


Format 

Speed 

(in  minutes) 

Accuracy 
(X  Errors) 

I.  Printed  Words 

45.9 

25 

II.  Printed  Words /Related  Pictorial 

38.8 

32 

III.  Pictorial 

31.2 

34 

IV.  Pictorial/Related  Words 

30.7 

24 

Note.  From  Booher  (1975). 
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Ly  pictorial,  related  printed  words  format 
Eully  procedurallzed  Instructions. 


Figure  9.  Highly  printed  words,  related  pictorial  format  for 
fully  procedurallzed  Instructions. 


JPA  Selection  Algorithms 


Another  application  for  the  JPA  organizing  concept  is  in  organizing 
the  various  algorithms  or  models  for  selecting  and  designing  optimum  perfor- 
mance aids.  In  doing  so,  it  appears  that  the  model  of  Poet  (1977a)  on  formats 
and  media  and  that  of  Shrlver  and  Hart  (1975)  on  fundamental  elements  are 
system-level  concepts,  while  the  user-data  match  model  developed  recently 
by  the  Navy  Technical  Information  Presentation  Program  (NTIPP)  (Hughes, 

1977b)  is  directed  specifically  to  the  component  level.  A selection  and 
organizing  model  being  developed  by  Bell  Laboratories  (Dever,  Note  5)  may 
fit  best  at  the  element  level. 

In  selecting  and  designing  optimum  performance  aids  there  are 
several  problems  that  seem  unique  to  each  level  and  that  should  be  emphasized 
in  their  respective  algorithms.  At  the  system  level,  for  example,  the  funda- 
mental elements  that  make  up  front-end  analysis  are  extremely  Important. 

Shriver  and  Hart  (1975)  pointed  out  that  in  designing  an  FPJPA  it  is  not 
enough  to  do  the  engineering  analysis  or  even  Che  human  factors  task  analysis. 
If  troubleshooting  is  involved,  then  a functional  analysis  is  required  as 
well;  and  if  the  JPA  is  Co  be  used  by  inexperienced  personnel,  then  a be- 
havioral task  analysis  is  also  required.  Table  8 Illustrates  the  different 
levels  of  front-end  analysis  that  are  Involved,  depending  on  the  particular 
format  and  content  selected  at  the  system  level.  But  even  these  analyses 
are  Inadequate  for  an  Integrated  JPA/tralnlng/job  design  system.  In  the 
JPA  conference  at  NAVPERSRANDCEN  (Booher,  1977a),  it  was  noted  that  trade- 
off analyses  among  the  various  disciplines  supporting  weapon  system  operation 
and  maintenance  must  also  be  conducted  as  early  as  possible  in  the  system 
development.  These  analyses  should  then  be  reiterated  throughout  the  system 
life  cycle.  The  earlier  and  the  more  completely  that  cost/performance  analyses 
are  done,  the  more  likely  it  is  that  maximum  benefits  will  accrue  from  each 
of  the  three  major  disciplines. 

General  rules  of  information  presentation  are  best  applied  at  the 
JPA  system  level.  Post  and  Price  (1974),  in  applying  their  organization 
principles  to  maintenance  task  technical  information  manuals,  recommended 
that  designers  should  also  attempt  (1)  to  develop  a maintenance  manual 
on  a hierarchy  of  maintainable  units,  (2)  to  Include  a repair  cycle  over- 
view that  gives  a diagrammatic  representation  of  the  succession  of  mainten- 
ance actions,  (3)  to  consolidate  all  information  concerning  a single  main- 
tainable unit  into  a single  package,  and  (4)  to  standardize  the  content, 
format,  and  sequence  of  information  within  a package. 

An  algorithm  for  selecting  optimum  job  aids  at  the  component  level 
is  considerably  different.  Often,  the  designer  will  have  an  opportunity 
at  the  presentation  component  level  to  choose  from  several  competing  formats 
for  conveying  the  information  relevant  to  certain  tasks.  The  choice  could 
be  within  the  print  medium  (i.e,,  table,  chart,  prose,  or  pictorial),  or 
it  might  be  among  audio  or  audiovisual  media.  Decisions  at  thin  level  vary 
V7ith  weapon  systems  and  tend  to  be  best  made  by  the  program  manager,  pre- 
ferably with  the  assistance  of  presentation  principles  guidelines.  Aids 
at  this  level  tend  to  be  structured  into  composite  wholes  capable  of  being 
presented  within  the  user^s  field  of  view.  Several  attempts  have  been  made 
to  classify  the  various  presentation  components  (Curran,  Note  6;  Powers, 
Dickerson,  & Nugent,  1977;  Hughes,  1977b). 
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The  question  of  which  JPA  (e.g.,  table,  cliart,  schematic)  Is  optimum 
for  presenting  fault  Isolation  Information  can  also  be  asked  best  at  the 
component  level.  However,  existing  data  are  generally  scarce  and  Inadequate 
for  making  simple  comparisons  of  JPA  presentation  components.  Any  decision 
about  what  Is  the  best  presentation  component  for  any  particular  task  and 
for  any  particular  personnel  attribute  must  be  extrapolated  from  other  data 
or  "expert  judgments."  Hughes  (1977b),  for  example.  In  developing  extremely 
elaborate  matrices  of  task  actions  versus  "presentation  components"  f^r 
several  Navy  ratings,  was  unable  to  cite  a single  "Ideal"  presentation 
component  for  any  specific  task  based  upon  data  from  controlled  experiments. 
Specifications  and  guidelines  for  cou^onent  selection  for  a particular 
task  are  also  generally  unavailable  at  this  level. 

As  of  this  writing,  the  Bell  Laboratories  study  (Dever,  Note  S) 
was  not  available  for  review.  According  to  Dever* s description,  however. 

Bell's  structured  programming  technique  appears  to  be  useful  as  an  algorithm 
at  the  element  level.  It  has  been  observed  that  many  of  the  "comprehensibility" 
problems  facing  the  designer  of  computer  programs  are  similar  to  those  of  job 
aid  designers.  Consequently,  much  of  what  Is  already  known  about  writing  com- 
puter programs  can  be  applied  to  procedural Ized  job  aids. 

Theoretical  Trends 


The  JPA  organizing  concept  may  also  help  to  group  the  theoretical 
trends  most  applicable  to  performance  aiding.  It  seems  that  the  literature 
on  motivation  theory  as  applied  to  work  and  work  situations — theories  of  job 
enrichment,  job  enlargement,  and  vertical  and  horizontal  loading— are  most 
applicable  at  the  system  level.  Collins  (1977)  suggested  that  the  experiential 
learning  model  and  the  action  research  method,  with  Its  foundation  In  Lewlnlan 
theory,  can  be  suggestive  of  new  developments  In  performance  aid  technology. 
These  newer  concepts,  as  well  as  older  ones  such  as  those  of  Miller,  Galanter, 
and  Pribram  (1960) , show  how  people  develop  hierarchical  plans  and  strategies 
In  organizing  and  executing  their  activities,  and  can  best  be  applied  to  the 
Integration  of  JPA  systems  Into  the  larger  organization.  For  example,  the 
theory  of  Miller  et  al.  suggests  that  people,  while  welcoming  the  organiza- 
tion of  Information  they  must  filter  to  plan  their  own  activities,  object 
to  being  tightly  regimented  In  planning,  organizing,  and  executing  their  own 
behaviors.  In  procedurallzed  troubleshooting  this  objection  to  rigidity 
might  be  expected  to  Increase  from  the  novice  technician  who  welcomes  the 
procedure  (because  there  Is  no  other  way  he  could  do  the  job)  to  the  experi- 
enced troubleshooter  who  might  take  serious  exception  to  a particular  pro- 
cedure (because  of  the  Increased  demands  or  because  It  does  not  allow  for 
his  greater  knowledge  of  possible  system  faults) . This  points  out  the 
necessity  for  future  JPA  systems  to  be  sufficiently  flexible  to  change  In 
response  to  personnel  career  progression. 

At  the  coiiq>onent  level,  the  theoretical  and  experimental  base  seem 
to  be  rooted  most  flirmly  In  Information  communication  theory.  The  work  done 
In  education  and  training  on  audiovisual  channels  of  connunlcatlon  Is  applic- 
able, particularly  when  organized  under  Hartman's  (1961)  model  for  multiple- 
and  single-channel  communication.  Hartman  said  that,  since  the  advent  of  motion 
pictures  and  television,  researchers  In  communication  have  tended  to  define 
channels  according  to  the  kind  of  Information  presented.  In  multiple-qhannel 
communication,  the  different  channels  may  present  redundant  Information, 


related  information,  unrelated  information,  or  contradictory  information. 

In  JPA  design,  multiple  channels  are  desired  only  if  the  information  in  the 
two  channels  can  be  presented  in  redundant  or  related  fashion,  and  with  a 
high  degree  of  confidence  in  low  Interference  between  channels. 

In  single-channel  presentations,  performance  on  tasks  aided  by  visual 
formats  has  been  found  to  be  superior  or  equivalent  to  tasks  utilizing 
audio  formats  (Elliott,  1966;  Severin,  1967;  Goff  et  al.,  1969).  Written 
formats  are  more  accurate  than  pictorial  (Dv^er,  1968;  Booher,  1975)  but 
pictorial  formats  take  less  time  to  use  (Dwyer,  1968;  Dickey  & Schneider, 

1971;  Booher,  1975).  The  usefulness  of  pictorial  information  in  facilitating 
performance  in  concrete  operational  tasks  appears  to  be  relatively  indepen- 
dent of  personnel  attributes.  Ir  one  experiment  with  Navy  maintenance  per- 
sonnel from  10  job  skill  categories,  the  best  performance  was  achieved  when 
the  pictorial  channel  was  used  as  the  primary  channel  of  a multiple-channel 
format  (Booher,  1975).  Similarly,  studies  with  college  students,  both  hear- 
ing (Dwyer,  1968,  1972)  and  hearing-impaired  (Reynolds  & Rosen,  1973),  have 
shown  advantages  in  the  comprehension  of  individualized  narrative  presenta- 
tions with  the  use  of  highly  pictorial  formats. 

Lewis  and  Cook  (1969),  in  their  theory  of  telling  (which  emphasizes 
one-way  communication  between  the  user  and  the  performance  aid),  could  be 
useful  to  investigators  at  both  the  component  and  element  levels.  In  their 
words,  "A  theory  of  telling  might  be  equivalently  described  as  a theory 
of  clear  communication.  Ideally,  it  would  specify  the  conditions  under  which 
information  could  be  communicated,  in  one  direction  only,  without  ambiguity. 

It  would  spotlight  the  difficulties  that  frustrate  the  unambiguous  transfer 
of  information,  and  it  would  serve  as  a reminder  that  problems  of  communication 
are  problems  of  unclear  communication." 

Chapanis  (1965) , in  addressing  the  problem  of  following  simple  instruc- 
tions, pointed  out  how  few  data  are  available  for  decisions  that  might  be 
made  in  designing  Instructions.  At  that  time  he  was  able  to  find  only  one 
experimental  study  that  addressed  the  problem  of  the  understandablllty  of 
the  information  being  presented.  A clearly  formulated  theory  of  telling 
that  concentrates  on  understandablllty  might  be  most  beneficial  for  guiding 
further  JPA  research. 

One  of  the  most  perplexing  research  problems  at  the  presentation  com- 
ponent and  element  levels  is  that  of  controlling  content.  It  is  very  dif- 
ficult to  ensure  that  content  does  not  change  when  fonnats  are  varied.  In 
fact,  much  of  the  research  in  multiple-channel  comparisons  has  been  deficient 
in  this  area.  Many  examples  can  be  found  in  the  literature  where  one  type 
of  audiovisual  display  was  compared  with  another,  but  the  results  were  value- 
less because  totally  different  Information  was  presented  in  each  display. 

At  the  JPA  element  level,  experimenters  and  practitioners  have  come 
to  realize  that  there  are  serious  limitations  in  extrapolating  from  the 
large  experimental  data  base  on  perception,  memory,  learning,  retention, 
motivation,  readability,  and  so  forth  to  the  composite  wholes  (presentation 
coiiq)onents)  that  are  dictated  by  realistic  operation  and  maintenance  tasks. 
Since  the  data  were  seldom  collected  on  tasks  resembling  operator  or  main- 
tainer  tasks,  extrapolations  require  considerable  faith  that  the  job  task 
being  presented  in  a JPA  component  can  be  adequately  defined  by  a summation 
of  behavioral  subtasks. 
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Several  researchers  (Newman,  1957;  Folley,  1961)  attempted  to  Identify 
potential  JFA  techniques  through  a taxonomy  of  behavioral  suhtasks,  but  this 
does  not  appear  to  be  a particularly  fruitful  approach.  The  listing  of  each  . 
subcask  becomes  an  extremely  complex  way  of  defining  the  various  behaviors 
Involved  in  job  performance.  Also,  the  data  linking  Che  various  behavioral 
subtasks  to  specific  job  aid  formats  are  extremely  limited,  and  the  complex 
interactions  of  behavioral  subtasks  for  any  realistic  tasks  are  undefined. 
Studies  that  have  succeeded  In  finding  optimum  formats  for  realistic  tasks 
(Dwyer,  1968,  1969;  Booher,  1975)  have  varied  format  across  wide  dimensions 
while  maintaining  fairly  gross  categories  of  behavioral  tasks.  Furthermore, 
even  if  these  problems  could  be  overcome,  the  final  JFA  format  designed  to 
optimize  each  behavioral  sub task  would  probably  be  rejected  either  as 
being  too  voluminous  or  too  densely  compacted,  depending  on  the  degree  of 
procedurallzatlon  required  from  earlier  decisions  at  the  JPA  system  level. 

^ An  optimum  format  size  for  guiding  operator  malntainer  actions 

can  usually  be  assumed  to  lie  somewhere  near  the  totally  verbal  print  or 
< the  completely  schematic.  Any  deviation  from  these  two  extremes  will  gen~ 

erally  require  greater  space  and  be  relatively  taore  difficult  to  prepare, 
thereby  Increasing  the  size  of  the  format  and  cost.  Decisions  to  carry 
optimization  to  the  level  of  the  behavioral  subtasks  must  be  approached 
cautiously,  therefore,  with  these  factors  In  mind. 

C 

Perhaps  a totally  new  theoretical  approach  Is  needed  at  the  element 
level.  Concepts  In  memory  would  seem  to  be  fundamental  to  an  understanding 
of  optimum  ways  to  organize  Information  at  this  level.  Smlllle  (Note  1),  In 
I a recent  doctoral  dissertation  on  performance  aiding,  used  memory  concepts 

j such  as  encoding  specificity,  cue-dependent  forgetting,  episodic  memory, 

I and  depth  of  processing  to  explain  past  research  findings;  but  concepts 

1 in  memory  (along  with  theories  of  perception  and  reading)  have  yet  to  be  used 

I satisfactorily  in  formulating  and  testing  hypotheses  for  performance  aid 

research. 


NEw  DIRECTIONS  IN  PERFORMANCE  AIDING  RESEARCH 


Use  of  Memory  in  JPAs 

The  theoretical  discussion  of  JPAs  by  Erase  (1976)  and  Smillie  (Note  1) 
may  be  beneficial  for  researchers  and  JPA  designers  at  the  presentation 
component  or  element  levels.  Fraae  described  a model  of  the  information 
processing  skills  invoxved  in  reading,  with  an  emphasis  on  memory  in  reading 
performance.  Smillie  suggested  that  theoretical  developments  in  memory  be 
used  for  hypothesis  formulation  and  testing  of  JPAs.  Furthermore,  Smillie 
concluded  that  the  most  important  concepts  contrlbid.^ing  to  a theoretical 
base  for  understanding  JPA  performance  differences  in  print  and  pictorial 
channels  are  (1)  pictorial  superiority,  (2)  encoding  speclficity/cue-depen- 
dent  forgetting,  (3)  episodic  memory,  and  (4)  depth  of  processing. 

There  is  considerable  agreement  among  researchers  that  pictorial  stimoli 
are  processed  faster  and  remembered  better  than  their  verbal  equivalents. 
Numerous  explanations  have  been  made  to  account  for  this  fact.  In  stimulus- 
response  terms,  Klatsky  (1975)  concluded  that  processing  is  faster  for  pictorial 
stimuli  because  the  process  of  forming  symbolic  representations  from  verbal 
Inputs  is  bypassed.  Other  explanations  for  the  relative  superiority  of  the 
pictorial  channel  are  (1)  that  the  array  search  time  is  reduced  (Pavlo  & Begg, 
1974;  Handler  & Johnson,  1976),  (2)  pictorial  material  may  be  encoded  both 
aurally  and  visually  (Pellegrino,  Siegel,  & Dhawan,  1975,  1976),  and  (3) 
the  sensory  code  for  pictures  seems  to  be  more  differentiating  and  less 
susceptible  to  Interference  from  successive  items  (Nelson,  Reed,  & Walling, 
1976).  This  does  not  mean,  however,  that  pictorial  .IPAs  will  necessarily 
be  comprehended  better  than  verbal  JPAs.  There  are  other  aspects  of  the 
storage  and  retrieval  process  that  have  to  be  considered  before  any  such 
generalizations  can  be  drawn. 

Tulvlng  (1974)  and  Tulving  and  Thomson  (1973)  discussed  the  perceptual 
encoding  and  forgetting  aspects  of  storixig  information  for  later  retrieval. 

They  noted  that  the  retrieval  process  depends  on  retrieval  cues  that  are 
moot  precise  in  accessing  previously  stored  information.  The  availability 
of  retrieval  cues  in  memory  depend  upon  (1)  how  well  during  encoding  the 
performer  perceives  the  cues  necessary  for  later  response,  and  (2)  how 
resistant  the  relevant  cues  are  to  forgetting.  JPAs  designed  to  take 
advantage  of  this  memory  process  would  strive  to  provide  cues  in  the  presen- 
tation format  that  (1)  are  most  likely  to  be  perceived  as  relevant  for  en- 
coding as  aids  for  later  information  retrieval,  and  (2)  are  most  likely  to 
trigger  retrieval  cues  at  the  time  of  performance.  Smillie  Implied  that 
specific  verbal  cues,  rather  than  pictorial,  would  lead  to  greatest  perfor- 
mance accuracy  under  this  concept. 

Episodic  memory  and  depth  of  processing  concepts  may  also  be  Important 
to  JPA  design  if  information  Inputs  are  truly  stored  as  unique  traces  (e.g., 
phonemic,  semantic,  Imaginal)  and  if  the  memory  traces  are  stored  as  specific 
episodes.  Smillie  Interpreted  this  to  mean  that  each  instruction  in  a proT 
cedurallzed  JPA  should  be  designed  to  provide  individuals  with  unique  episodes; 
that  is,  to  avoid  mlsculng  and  interference,  each  instruction  step  should  be 
as  distinct  as  possible  from  the  rest  of  the  instruction.  The  depth  of  pro- 
cessing of  each  unique  trace  tends  to  be  a function  of  the  amount  of  time 
and  the  level  of  analysis  undergone  while  attending  presentation  formats. 


It  Is  generally  easier  for  a designer  of  procedural  Instructions  to  provide 
unique  episodes  and  deeper  encodings  of  Information  with  verbal  formats. 

These  developments  In  memory,  when  looked  at  In  a performance  aiding  context, 
may  help  to  explain  why  both  pictorial  and  verbal  Information  must  be  used 
In  varying  combinations  to  maximize  performance  advantages. 

Towards  a Theory  of  JPA  Format  Mix 

The  theoretical  discussions  of  memory  and  cognition  and  of  multiple-  and 
single-channel  perceptual  modes  have  been  useful  In  Interpreting  unexpected 
findings  in  JPA  research,  but  they  have  not  been  particularly  useful  In 
predicting  optimum  format  design  or  In  formulating  JPA  research  hypotheses 
for  experimental  verification.  For  example,  the  cue  summation  and  stimulus 
generation  concepts  used  by  Hartman  (1961)  and  other  audiovisual  researchers 
to  resolve  discrepancies  In  findings  regarding  the  learning  efficiency  of  test 
materials  does  not  explain  the  results  of  JPA  picture-word  studies. 

What  seems  to  be  most  needed  at  all  JPA  conceptual  levels — system,  com- 
ponent, and  element — Is  a theory  for  mixing  JPA  techniques,  principles^ 
and  methodologies.  For  example.  It  is  now  thought  that  the  optimum  tech- 
nique at  the  system  level  may  be  to  combine  elements  of  both  the  FPJPA 
and  the  more  convevitlonal  deductive  aids  Into  a hybrid  aid  (Post,  1977a; 

Klesch,  1977).  It  is  also  Interesting  to  note  that  "pure"  presentation 
formats  (l.e.,  prose,  tables,  pictures,  and  schematics)  are  seldom  used 
alone  to  present  Information  for  realistic  tasks.  Printed  prose  or  pro- 
ceduralized  Instructions  usually  refer  to  cables  and  illustrations  for 
completing  the  information;  even  schematics  will  have  some  additional  non- 
schematic  Information  such  as  verbal  cues  or  wave  forms.  In  general,  the 
knowledge  that  performance  can  be  improved  by  combining  print,  pictorial, 
and  audio  channels  Is  not  nearly  so  valuable  to  JPA  design  as  knowing  what 
Information  Co  put  In  each  channel  and  how  to  structure  each  channel  to  be 
compatible  with  the  other. 

Recent  experiments  on  novel  Information  transfer  techniques  for  aiding 
location,  comparison,  and  symptom  recognition  tasks  (Bopher,  1975),  three- 
dimensional  performance  tasks  (Frey,  1976),  and  long-term  retention  (Sitterley, 
1974)  suggest  that  major  Improvements  In  JPAs  resulting  from  changes  In  the 
structure  of  the  format  itself  (as  opposed  to  content  change)  are  most  likely 
to  be  found  In  designs  that  primarily  address  the  human  visual  memory  system. 
However,  the  verbal  channel  must  usually  be  kept  available  In  a supporting 
role  because  of  the  ambiguity  present  In  most  techniques  that,  attempt  to 
convey  performance  Information  through  visual  memory.  Also,  the  visual  mode 
Is  generally  considered  Inferior  to  the  verbal  In  conveying  abstract  concepts, 
but  even  this  does  not  appear  true  for  all  people.  Split-brain  research 
Indicates  that  a high  level  of  abstract  reasoning  can  be  carried  on  In  the 
visual  memory  centers  (Diamond  & Beaumont,  1974). 

A start  toward  a theory  of  JPA  format  mixing  may  now  be  possible  for 
the  presentation  component  and  element  levels.  Although  a complete  theory 
Is  beyond  the  scope  of  this  task,  the  experimental  findings  of  JPA  investi- 
gators are  beginning  to  suggest  a theory  for  JPA  mix.  Booher  (1975)  obtained 
the  surprising  results  that  the  print  channel  alone  was  superior  In  perfor- 
mance accuracy  to  formats  that  most  closely  resemble  the  FPJPA,  and  that  the 
FPJPA  format  was  generally  inferior  In  both  performance  time  and  accuracy  to 
multiple-channel,  highly  pictorial  formats.  As  Booher  explained  It: 
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The  findings  of  relatively  poor  accuracy  and  time  performance 
with  high-print  multiple-channel  formats  suggest  that  the 
formats  which  most  closely  resemble  conventional  procedur- 
alized  instructions  are  not  structured  sufficiently  well 
to  take  advantage  of  the  distinct  pictorial  and  verbal 
Internal  processes  available  for  selecting,  translating, 
and  organizing  information,  (pp.  275-276) 

These  findings  now  appear  to  be  more  universally  applicable  than  was 
thought  at  that  time.  The  same  relative  performance  was  also  attained  with 
hearing-impaired  college  students  (Booher  & Reynolds,  in  press).  The  find- 
ings here  are  even  more  surprising  (i.e.,  the  relative  superiority  of  print 
format  in  performance  accuracy  over  pictorial  aid  FPJPA  formats)  because 
hearing  impairment  usually  seriously  degrades  reading  ability. 

A different  model  than  that  proposed  by  Hartman  (1961)  or  Smillie  (Note  1) 
appears  necessary  to  explain  these  unexpected  results.  It  should  first  be 
recognized,  as  suggested  by  Haber  (1970),  that  there  are  two  different 
processing  paths — pictorial  and  verbal — through  the  stages  of  selecting, 
translating,  and  organizing  information  presented  visually.  These  paths 
can  operate  either  independently  or  together,  depending  upon  the  informa- 
tion presented  and  the  Individual’s  approach  to  information  comprehension. 
Applying  this  to  JPA  comprehension,  it  can  be  assumed  that  the  pictorial 
path  translates  directly  from  pictorial  inputs  to  the  selection  of  a range 
of  possible  perceptual-motor  routines  for  use  in  conpletlng  the  task  that 
organizes  the  routines  in  a time-sharing  manner.  The  verbal  path,  however, 
translates  from  verbal  inputs  in  a more  step-by-step  manner,  tending  to  select 
routines  in  series.  It  is  Important  to  note  that  the  human  organism  is  capable 
of  using  both  together  in  an  optimum  fashion  but  is  forced  to  use  one  or  the 
other  initially  depending  on  what  type  of  format  is  presented. 

Before  these  experiments,  the  manner  in  which  people  are  most  predisposed 
to  use  the  two  systems  for  comprehending  mixed  formats  vras  not  known.  How- 
ever, these  results  strongly  suggest  that  people  understand  instructions  best 
when  the  pictorial  mode  is  used  to  aid  in  selecting  and  organizing  a range 
of  perceptual  motor  actions  and  when  verbal  material  is  available  to  confirm 
specific  actions  within  the  range. 

Another  way  of  looking  at  efficiency  in  processing  information  throu.gh 
mixed  formats  is  with  the  analogy  of  a two-stage  filtering  system:  a first- 
stage,  wlde-band  filter  and  a second-stage,  narrow-band  filter.  The  pictorial 
channel  would  be  best  for  the  wide-band  activity,  quickly  reading  the  manory 
routines  most  likely  to  be  applicable  to  the  situation  at  hand,  while  the  verbal 
channel,  acting  as  the  narrow-band  filter,  would  be  most  useful  in  selecting 
the  most  accurate  response  routine  available  within  the  range  obtained  by  the 
first  filtering  action. 

This  concept  may  have  applicability  beyond  the  proceduralized  instruction. 

A recent  study  (Curran  & Mecherikoff,  1978)  of  Navy  inaintenance  personnel 
performing  location  and  Identification  tasks  using  technical  manual  illustra- 
tions suggested  that  the  concept  may  also  apply  to  deductive,  nonprocedurallzed 
aids.  The  study  investigated  the  optimum  arrangement  and  design  of  callouts 
consisting  either  of  numbers,  nomenclature,  or  nomenclature  and  numbers  com- 
bined to  aid  in  locating  and  identifying  mechanical  parts  on  cross-sectional 


Illustrations.  In  discussions  before  the  data  collection,  an  attempt  was 
made  to  predict  the  optimum  callout  arrangement  and  mix  of  pictorial  and 
verbal  channels.  It  was  reasoned  that  a wlde-band  filtering  action  would 
first  be  required  to  get  the  user  to  the  correct  area  of  the  Illustration. 

This  could  best  be  done  with  callout  numbers  and  an  arrow  (a  highly  pictorial- 
channel  technique).  Often,  however.  It  Is  not  obvious,  simply  from  viewing 
the  illustration,  exactly  what  the  part  is.  A name  (verbal  channel)  would 
be  best  to  perform  such  identifications.  In  deciding  then  on  an  optimum 
format  for  callouts  to  aid  in  both  locating  and  identifying  parts,  the  theory 
suggests  that  the  designer  (1)  use  both  callout  numbers  and  arrows  (pictorial) 
and  names  (verbal),  and  (2)  arrange  the  numbers  sequentially  to  be  prominently 
displayed  alone,  with  each  name  nearest  its  own  callout  number  and  arrow. 

Curran  and  Mecherlkoff  found  that  nomenclature  and  number  callouts  arranged 
as  described  above  resulted  In  the  best  performance  for  Identification  tasks 
without  degrading  performance  on  location  tasks.  Some  very  useful  future 
research  efforts,  suggested  by  these  studies,  could  have  both  theoretical 
and  practical  merit.  A particularly  Interesting  research  area  would  be  the 
optimum  mix  of  verbal  and  pictorial  Information  for  schematics,  functional 
block  diagrams,  and  wiring  diagrams  used  in  maintenance  troubleshooting 
tasks.  Here  again,  the  problem  Is  not  so  much  when  to  use  a particular 
visual  aid  or  even  how  to  arrange  the  lines  and  blocks  on  a page.  Vfhat  the 
JPA  designer  needs  to  know  is  what  kind  of  verbal  Information  would  best  aid 
troubleshooting  and  where  should  it  be  placed  in  the  chosen  JPA  formats. 

Priorities  for  Future  JPA  Research  and  Technology 

In  R&D  efforts  at  the  system  level,  the  greatest  potential  payoffs  are 
seen  in  the  efforts  recommended  at  the  JPA  conference  (Bcoher,  1977)  and 
categorized  as  (1)  JPA  technology  application,  (2)  methodological  studies, 
and  (3)  future  R&D.  The  current  project  plans  to  address  the  first  two 
categories,  including  the  R&D  necessary  for  a long-term  test  and  cost/benefit 
evaluation  of  performance  aids  In  large  personnel  and  training  systems.  Two 
of  the  papers  presented  at  that  conference  (Collins,  1977;  Shriver,  1977), 
provided  specific  recommendations  for  future  JPA  R&D  at  the  system  level 
that  the  current  project  does  not  address  directly:  (1)  initiate  research 
and  development  efforts  to  fill  performance  data  gaps  In  the  JPA  technology 
base  and  to  explore  the  feasibility  of  new  concepts  for  JPA,  and  (2)  develop 
a scientific  and  technical  approach  for  Identifying,  developing,  testing, 
and  evaluating  principles  and  methodology  applicable  to  the  Implementation 
of  advanced  technology  Into  Navy  weapon  and  logistics  support  systems. 

Research  should  be  conducted  at  the  systems  level,  therefore,  with  JPA 
system  technolo;^y  remaining  relatively  fixed  while  the  effects  on  training, 
the  personnel  system,  and  job  design  variables  are  studied.  In  general, 
the  most  applicable  methodology  for  Implementing  an  Integrated  JPA/trainlng/ 
job  design  paradigm  should  be  that  of  an  integrated  personnel  system,  capable 
of  self-design,  test,  evaluation,  and  change  in  response  to  advances  in  tech- 
nology. Unfortunately,  an  integrated  personnel  system  implementation  metho- 
dology does  not  currently  exist,  and  efforts  along  these  lines  may  need  to 
look  to  a project  such  as  the  job  aids  T&E  for  the  data  necessary  for  Its 
own  development. 
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More  specifically,  research  on  work,  motivation,  learning,  and  Issues 
of  aiding  versus  other  technological  means  for  Improving  performance  are 
especially  important  system  considerations  in  successful  JPA  Implementation. 

The  methods  and  problems  of  integrating  JPAs  with  training  and  job  design 
should  receive  immediate  attention,  particularly  with  efforts  in  algorithm 
development.  Also,  problems  of  career  advancement,  skills  retention,  and 
other  major  personnel  system  factors  have  not  really  been  investigated 
for  the  effects  of  progressively  changing  JPA  systems  (e.g.,  FPJPA  to  hybrid 
to  deductive).  The  optimum  points  in  the  personnel  career  paths  for  the 
introduction  of  higher  level  JPAs  should  probably  be  another  major  factor 
to  consider  in  future  algorithms  and  integrated  personnel  system  trade-off 
models. 

In  addition,  there  is  still  a need  to  determine  the  effects  of  aiding 
on  lowered  aptitude  levels  and  to  study  the  motivation  to  use  aiding  systems. 
Studies  at  the  other  two  JPA  conceptual  levels  may  give  some  Information  about 
the  ability  of  different  aids  to  allow  lower-aptitude  personnel  to  perform 
better  or  to  provide  greater  job  satisfaction  through  on-the-job  performance, 
but  whether  performance  aids  are  to  be  used  more  than  5 percent  of  the  time 
(as  is  generally  reported  for  maintenance  Instructions)  or  whether  lower- 
aptitude  personnel  can  handle  other  aspects  of  the  job  that  have  not  been 
aided  are  systems  research  problems. 

At  the  JPA  component  level,  work  on  new  aiding  devices,  basic  comparison 
performance  data,  and  further  work  on  aids  that  change  with  personnel  needs 
(e.g.,  hybrid  aids)  is  still  warranted.  Most  needed  at  this  level  are  criteria 
for  making  valid  comparisons  that  consider  both  cost  and  performance  data.  The 
biggest  problem  here  is  obtaining  valid  data  on  what  it  costs  to  develop  a 
JPA. 


The  research  problems  at  the  JPA  element  level  are  greatly  in  need  of 
basic  research.  A whole  new  approach  may  be  required  to  solidify  the  founda- 
tion of  performance  aiding.  The  readability  of  technical  instructions,  for 
example,  needs  something  more  than  standards  of  short  words  and  short  sentences 
drawn  from  the  reading  research  for  education.  Performance  data  on  the  com- 
prehensibility of  technical  instructions  Indicate  that  Information  transfer 
processing  for  decision  making  is  on  the  order  of  5 to  25  words  per  minute, 
whereas  reading  studies  operate  more  on  the  order  of  100  to  1000  words  per 
minute. 

Finally,  a problem  that  has  yet  to  be  addressed  and  that  cuts  across 
all  three  JPA  levels  is  that  of  determining  what  kind  of  process  is  needed 
to  produce  and  measure  high-quality  aids.  If  performance  aids  cannot  be 
made  extremely  accurate  at  a reasonable  cost,  then  the  attempts  to  solve 
systems  pr'^blems  in  motivation  and  productivity  with  lower  skill  and  experi- 
ence levels  are  not  going  to  be  very  satisfactory.  Similarly,  holograms 
with  false  images  or  picture-word  formats  with  erroneous  illustrations  and 
verbal  descriptions  are  of  little  value  to  the  user. 

Perhaps  what  is  needed  most,  whether  for  improving  performance  aids, 
finding  management  techniques  to  raise  quality,  or  compensating  for  in- 
adequate quality,  are  early  warning  systems  with  methods  of  predicting 
errors  in  the  aid  development  process. 
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CONCLUSIONS 


Several  JPA  systems,  classified  as  format/content  types,  are  sufficiently 
developed  to  be  Integrated  with  major  military  personnel  and  training  systems 
with  reasonable  expectations  of  major  payoffs  In  reduced  personnel,  training, 
and  maintenance  costs  and  In  Increased  overall  operational  readiness.  These 
basic  types  of  JPAs  vary  primarily  in  accordance  with  the  amount  of  pro- 
ceduralization  required,  which  in  turn  Is  directly  proportional  to  the  amount 
of  front-end  analysis  employed.  The  major  cost  Items  for  comparing  one  format/ 
content  JPA  with  another  are  the  amount  of  detail  required  for  front-end 
analysis  and  the  amount  of  required  graphic  or  verbal  enrichment.  The 
greater  the  amount  of  procedurallzatlon  and  enrichment,  the  larger  the  size 
of  the  aid. 


Three  format/content  types — the  FPJPA,  the  hybrid  aid,  and  the  deductive 
aid — seem  sufficient  to  cover  the  aiding  needs  of  most  operators  and  maln- 
talners  throughout  their  Navy  careers.  With  the  exception  of  quality  assur- 
ance, which  is  considered  a delivery  system  problem,  the  FPJPA  and  the  deduc- 
tive aid  (NAVAIR  Work  Package  or  NAVSEA  SIMM/FOMM)  are  fully  developed 
systems.  The  hybrid  aid  Is  still  in  conceptual  form,  and  probably  needs 
further  refinement  before  full-scale  demonstration  or  implementation. 

This  is  particularly  true  for  complex  digital  electronic  systems  where  no 
satisfactory  aiding  techniques  were  found. 

Three  other  major  types — display  media,  applied  training,  and  peripheral 
diagnostic — are  secondary  means  of  improving  performance  through  aiding. 
Systems  that  could  be  classified  into  these  groups  often  had  a format/con- 
tent aspect  that  accounted  for  the  increases  in  performance. 


The  greatest  weakness  In  JPA  systems  technology  now  lies  in  the  systems 
for  producing,  distributing,  and  updating  accurate  JPAs.  The  only  delivery 
systems  presently  available  for  providing  format/content  JPAs  are  improved 
technical  publication  systems.  The  Navy  Technical  Information  Presentation 
System  (NTIPS)  now  under  development  should  greatly  strengthen  this  branch 
of  the  technology. 

Unique  demands  of  the  Job  environment  (e.g.,  work  space,  wind,  temperature) 
often  create  special  physical  design  requirements  for  the  job  aid.  In  most 
instances,  however,  specially  developed  devices  (e.g.,  audiovisual  or  micro- 
form) offer  few  advantages  and  several  disadvantages  to  the  user.  Most  infor- 
mation aiding  needs  obtainable  within  the  state-of-the-art  can  usually  be  met 
with  hard-copy  print  modified  to  fit  environmental  constraints.  Devices  that 
can  inherently  Increase  the  capability  of  the  user  beyond  that  possible  with 
hard-copy  print  (e.g.,  three-dimensional  devices.  Interactive  coiiq)uter— aided 
devices)  need  further  research  to  determine  whether  or  not  aiding  technology 
can  be  advanced  along  these  lines. 

Much  of  the  recent  JPA  research  by  the  Army,  Navy,  and  Air  Force  has  been 
directed  toward  the  development  of  algorithms  or  models  to  be  used  in  select- 
ing the  optimum  JPA.  Several  factors  are  important  in  the  selection  of  the 
appropriate  aid  form.  In  most  Instances,  the  application  of  these  factors 
to  an  algorithm  for  the  selection  of  optimum  performance  aids  is  not  com- 
plicated. The  FPJPA,  for  all  its  promotion  as  a solution  to  the  problem  of 
decreased  enlistment  skills,  does  not  appear  satisfactory  for  aiding  lower- 
aptitude  personnel  to  do  troubleshooting.  However,  Inexperienced  personnel 
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who  meat  the  usual  aptitude  levels  can  do  troubleshooting  with  the  aid  of 
an  FPJPA  after  mlnioal  training.  Also,  the  FPJPA  will  not  satisfactorily 
cover  all  troubleshooting  and  maintenance  actions,  particularly  those  in 
complex  electronic  systems  (both  digital  and  analog).  Such  maintenance 
tasks  (perhaps  25%  of  the  maintenance  workload)  require  additional  training 
and  less  procedurallzed  aids. 

Once  a set  of  aids  for  a particular  weapon  system  or  a particular  rating 
has  been  selected,  the  preferred  medium  and  other  physical  characteristics 
must  be  determined.  Too  often,  such  decisions  are  made  first  and  other 
decisions  must  be  made  as  compromises.  Although  there  are  exceptions,  the 
general  rule  Is  to  provide  a hard-copy  print  medium  wherever  possible. 

Aiding  technology  stands  on  an  equal  footing  with  training,  but  the 
need  to  use  lower-aptitude  personnel  and  to  cope  with  the  high  costs  of 
training  makes  aiding  a much  more  attractive  technique  for  meeting  most  job 
objectives  with  first-term  enlistees.  Under  current  conditions  In  the  military. 
It  Is  more  reasonable  to  save  training  Investments  for  those  who  are  most 
likely  to  make  the  military  a career  and  to  emphasize  aiding  at  the  begin- 
ning of  enlistments.  Job  design  Is  different;  it  both  leads  and  follows  the 
selection  of  the  JPA.  In  leading,  it  helps  determine  the  actual  need  for 
aiding  and  how  to  plan  which  people  In  a work  force  should  be  aided,  trained, 
or  both.  It  follows  by  being  applied  In  the  work  centers  themselves.  If 
particular  JPAs  are  available,  then  the  work  center  can  plan  its  workload 
accordingly. 

A conceptual  organization  of  JPA  technology  has  long  been  needed,  as  Is 
evident  from  the  piecemeal  approaches  to  research  and  the  turnover  of  experts 
in  the  field.  Basic  research  is  obviously  needed  to  answer  questions  about 
the  behavioral  foundations  of  aiding.  There  Is  still  a need  to  determine 
the  effects  of  aiding  on  lowered  aptitude  levels  and  the  motivation  to  use 
aiding.  Cost/performance  criteria  and  basic  cost  data  are  needed  to  make 
valid  comparisons  of  aid?  In  trade-off  models.  Finally,  research  is  needed 
to  determine  what  kind  of  process  is  required  to  provide  and  measure  high 
quality  JPAs. 

After  20  years  of  development,  performance  aiding  Is  now  clearly  a sought- 
after  technology  with  clear  directions  for  Improvement,  But  it  is  not  unusual 
for  organizational  ineicia  to  limit  promising  advances  before  they  can  demon- 
strate their  full  potential.  It  remains  to  be  seen  If  the  military’s  personnel 
organizations  can  adjust  to  the  impact  that  will  accompany  the  Intx'oductlon 
of  such  a sweeping  technology  as  that  offered  by  Job  Performance  Aids. 


RECOMMENDATIONS 


1.  The  methods  and  problems  of  Integrating  JPAs  with  training  and 
job  design  should  receive  immediate  attention  in  efforts  directed  toward 
algorithm  development. 

2.  Problems  of  career  advancement,  skills  retention,  and  other  major 
personnel  system  factors  should  be  investigated  to  determine  the  effects  of 
progressively  changing  JPA  systems. 

3.  Task-oriented  training  and  job  design  should  be  developed  to  support 
or  complement  the  aiding  system. 

4.  Aiding  as  a personnel  system  technology  should  act  on  an  equal 
footing  with  selection  and  training,  but  it  should  be  the  primary  technique 
for  meeting  most  job  objectives  for  new  or  first-term  enlistees. 

5.  Front-end  analyses  commonly  conducted  for  Integrated  JPA  and  train- 
ing requirements  should  be  expanded  to  include  job  design  requirements. 

6.  Selection  of  media  other  than  hard-copy  print  should  be  based  on 
system  support  requirements  that  Improve  the  timeliness  or  accuracy  of 
the  aid  rather  than  personnel  performance-related  criteria, 

7.  Development  of  new  aiding  devices  (e.g.,  hand-held  calculators, 
holographic  displays),  aids  that  change  with  personnel  needs  (e.g.,  hybrid 
aids),  and  aids  that  specifically  address  digital  electronic  systems  are 
needed  to  fill  gaps  and  to  advance  the  state-of-the-art  in  aiding  tech- 
nology. 
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